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In the United States today, more jobs than ever before require at least some postsecondary education. 
Yet too many young adults are either not enrolling or not succeeding in college. This scenario exists 
across many different types of communities, but schools in rural areas, particularly those with large 
populations of low-income students, face unique challenges in preparing and inspiring students to 
attend college. To address these challenges, the AVID (Advancement Via Individual Determination) 
Center — a nonprofit organization working to close the achievement gap for minority and low- 
income students — partnered with three rural school districts and the local state college in central 
Florida to develop and implement programming focused on strengthening college preparedness 
among middle school and high school students. Supported by funds from an Investing in Innovation 
(i3) development grant from the U.S. Department of Education, the partners worked together to 
implement the AVID College Readiness System (ACRS) across eight schools, train secondary and 
postsecondary instructors in a shared set of teaching strategies and best practices, strengthen the 
academic rigor of their classes, and develop a set of “alignment activities” for school staff members 
focused on collaboration and consistency of teaching and study strategies across middle school, 
high school, and college. 


The i3 grant also includes an evaluation, conducted by MDRC, of implementation and outcomes 
over the first three years of the project. The implementation study examines how closely the imple- 
mentation of the ACRS hewed to the model design and examines the drivers of and obstacles to its 
success. The outcomes study uses a “pre-post” nonexperimental method (which does not capture 
causation) to compare both school staff outcomes and student outcomes before implementation with 
outcomes during the implementation years to explore the promise of the system to positively affect 
schools and students. The report presents several key findings: 


e Overall, analyses show that the ACRS was implemented successfully at most schools with fairly 
high fidelity to the model. There was mixed success implementing the alignment activities. 


e Positive change was seen in teachers’ reported use of most ACRS teaching strategies, and in teach- 
ers’ and other staff members’ reported attitudes toward academic rigor and college preparation 
for all students and reported collaboration within and across grade levels and schools. 


e Little difference was found between the reported study habits and learning skills, engagement 
in school, and postsecondary expectations of students surveyed before implementation and of 
those surveyed after three years of exposure; however, on average, both groups had relatively high 
positive responses on most of the measures. 


e Students were more likely to take advanced courses, such as honors and Advanced Placement, 
and earned more credits in these courses, which are intended to strengthen their preparation for 
the rigor of college work. 


e Little difference in other measures of students’ academic performance (grade point average and 
English Language Arts standardized tests), educational attainment (overall credits earned and 
graduation), and high school persistence were found after three years of implementation compared 
with the outcomes before implementation. 


Building College Readiness Across Rural Communities | iii 


OVERVIEW iii 


eS, OL, ec a eae er ev eve ual 
Lil SL akc t oe co ee ee eee eee ene eee ee eee ee oe us 
PNAS Aah ale bl a eee ee a eee meee ee eee 
eA dD Nah || ee eR 
CHAPTER 
Introduction 1 
The AVID Central Collaborative Partners 2 
The Evaluation 3 
The ACRS and Alignment Activities 3 
Theory of Change 11 
Participating Districts and Schools 11 
Structure of This Report 14 
AVID Study Design 15 
Implementation of the ACRS and Alignment Activities 15 
School and Student Outcomes 18 
Implementation 23 
Fidelity of the AVID College Readiness System to the Model 24 
Fidelity of the Alignment Activities to the Model 29 
Factors That Enabled and Impeded Project Implementation 34 


School Outcomes: Adoption of AVID Teaching Methodologies, AVID Core 


Values, and Collaboration by School Staff 38 
Use of the WICOR Model 40 
AVID Core Values 40 
Collaboration A 
Conclusion 44 
Student Outcomes: Behaviors, Attitudes, and Academics 43 
Student Outcomes in Learning Skills, Engagement, and College Aspirations 44 
Student Academic Outcomes 48 
Conclusion 54 


Building College Readiness Across Rural Communities | v 


Conclusion and Reflections 55 


Summary of Findings 55 
Academic Rigor 56 
Communities of Practice for Teachers 57 
Dual Enrollment Versus Advanced Placement Courses 58 
College Collaboration 58 
APPENDIX 
Implementation Research Supplemental Materials 59 
Staff and Student Survey Supplemental Materials 63 
Student Academic Outcomes Supplemental Materials 81 
REFERENCES 92 


vi | Building College Readiness Across Rural Communities 


TABLE 


Grade Levels Supported by the AVID College Readiness System (ACRS), by 
Implementation Year 


Number of Schools Meeting Full Implementation Standard on Key 
Components and Elements of the AVID College Readiness System (ACRS), by 
Year 


Achievement of Implementation Standards on Key Components of the AVID 
Alignment Activities, by Year 


School Outcomes: Composite Staff Survey Measures, 
All Schools 


Student Survey Outcomes: Tenth-Grade Students, After Three Years of the 
AVID College Readiness System (ACRS) and Alignment Activities 


Student Outcomes in the Third Year of the AVID College Readiness System 
(ACRS) and Alignment Activities, Ninth- through Twelfth-Grade Students 
Combined 


Student Outcomes in the Third Year of the AVID College Readiness System 
(ACRS) and Alignment Activities, Eighth-Grade Students 


Enrollment in College in the Fall Following High School Graduation: 2014, 
2015, and 2016 High School Graduates 


Matrix 


AVID Core Values and Collaboration as Measured by the Staff Survey, by 
School, Year 3 


Student Survey Outcomes: Tenth-Grade Students in AVID Elective, After 
Three Years of the AVID College Readiness System (ACRS) and Alignment 
Activities 


Building College Readiness Across Rural Communities | vii 


TABLE 


Characteristics of Students in the Analysis Sample, Before and During the 
Third Year of the AVID College Readiness System (ACRS) and Alignment 
Activities 


Student Outcomes in the First Year of the AVID College Readiness System 
(ACRS) and Alignment Activities: Ninth- and Tenth-Grade Students 
Combined 


Student Outcomes in the Second Year of the AVID College Readiness System 
(ACRS) and Alignment Activities: Ninth, Tenth, and Eleventh Grades 
Combined 


Student Outcomes in the Third Year of the AVID College Readiness System 
(ACRS) and Alignment Activities, by Grade 


Student Outcomes in the Third Year of the AVID College Readiness System 
(ACRS) and Alignment Activities, by High School Attended 


Logic Model for the AVID College Readiness System (ACRS) and 
Alignment Activities 


Logic Model for the AVID College Readiness System (ACRS) and 
Alignment Activities 


The WICOR Model 


Key Roles in the ACRS and Alignment 
Activities Implementation 


viii | Building College Readiness Across Rural Communities 


Postsecondary credentials have become almost essential to successfully compete in the U.S. labor 
market, where technical and critical thinking skills are much prized. But many young adults, espe- 
cially from low-income families, are not acquiring these credentials — in part because many high 
school graduates across the United States are inadequately prepared for college-level course work. 
In addition, too often, low expectations for students in middle school, high school, and college leave 
some students struggling to succeed. Rural communities face particular challenges in ensuring college 
readiness and success for students. Given the distance from urban centers, geographic size, and low 
population density in these communities, colleges there often have trouble attracting and retaining 
skilled teachers and have less opportunity for collaboration across institutions. 


The AVID (Advancement Via Individual Determination) Center, a nonprofit organization working 
with school districts and schools in this country and abroad, is committed to closing the achieve- 
ment gap for minority and low-income students and ensuring that all secondary school students 
are prepared for success in college and their careers. The long-standing and widely used AVID 
elective, a cornerstone of the AVID Center’s approach, supports middle-achieving students (those 
earning Bs and Cs) in taking and passing rigorous college preparatory courses during middle school 
and high school. The AVID Center has lately expanded its focus to promote its teaching strategies 
and to foster strong learning behaviors among students schoolwide. In 2013, with support from an 
Investing in Innovation development grant, the AVID Center partnered with three rural Florida 
school districts and a local state college to design and implement a system intended to align curricula 
and teaching and learning strategies across institutions, to build a communitywide commitment to 
college readiness and success for all students, and to create opportunities for staff at secondary and 
postsecondary institutions to collaborate. 


After three years, most schools saw positive changes in staff’s reported use of the AVID teaching 
methodologies, expectations of students to do rigorous course work and prepare for college, and 
collaboration among staff and across schools. Students were more likely to earn credits in advanced 
courses, such as honors and Advanced Placement courses — presumably making them better prepared 
for college work — but little difference was observed in other measures of academic performance, 
educational attainment, and high school persistence. 


This report describes the findings from MDRC’s study of the implementation of the AVID system, 
including obstacles the partners faced and their successes. While it cannot speak to causation — the 
study it describes was nonexperimental, and therefore captures only associations — it examines the 
promise of a partnership like this one to positively affect student attitudes, academic achievement, 
and persistence in school. 


Gordon L. Berlin 
President, MDRC 
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ttaining some postsecondary education or a postsecondary credential is associated with 

achieving better employment outcomes and earning a living wage in the United States 

today. According to a 2016 report from the Center on Education and the Workforce, a 

majority of the 11.6 million new jobs that have been added to the economy since the 
Great Recession of 2007-2009 have gone to workers with at least some college education.’ However, 
less than half (46 percent) of 25- to 29-year-olds across the United States possessed an associate’s 
degree (or higher) in 2016.” One obstacle to degree attainment may be that many high school gradu- 
ates arrive at college unprepared for college-level course work. According to the National Center for 
Education Statistics, only 37 percent of high school seniors scored at or above proficient in reading 
and only 25 percent of seniors scored at or above proficient in math on the National Assessment of 
Educational Progress in 2015.° 


Rural middle schools and high schools, in particular, face unique challenges in preparing students 
for college and inspiring them to attend. These schools often serve large geographic areas with lower 
population density and a more limited tax base than those in urban or suburban settings, forcing 
them to stretch their resources further.* As a result, rural-area teachers are often asked to cover 
multiple classes or subjects, requiring more time to prepare and leaving less time to collaborate with 
their colleagues,® which is important for strengthening teaching and learning skills.® Given their 
distance from urban centers, many rural schools also face challenges in recruiting and retaining 
qualified teaching staff and in offering professional development and training.’ 


One response to the difficulties that rural-area schools face has come from the AVID (Advancement 
Via Individual Determination) Central Florida Collaborative, a group of educators and stakeholders 
who are focusing on building college preparedness among students in select middle schools and high 
schools in rural central Florida. To achieve its objective, the Collaborative implemented the AVID 


1. Anthony P. Carnevale, Tamara Jayasundera, and Artem Gulish, America’s Divided Recovery: College 
Haves and Have-Nots (Washington, DC: Georgetown University Center on Education and the Workforce, 
2016). 


2. Joel McFarland, Bill Hussar, Cristobal de Brey, Tom Snyder, Xiaolei Wang, Sidney Wilkinson-Flicker, 
Semhar Gebrekristos, Jijun Zhang, Amy Rathbun, Amy Barmer, Farrah Bullock Mann, and Serena Hinz, 
The Condition of Education 2017, NCES 2017-144 (Washington, DC: U.S. Department of Education, 
Institute of Education Sciences, National Center for Education Statistics, 2017). 


3. National Center for Education Statistics, U.S. Department of Education, The Nation’s Report Card: 2015 
Results, Mathematics & Reading, Grade 12 (Washington, DC: Institute of Education Sciences, 2015). 


4. Cynthia Reeves, /mplementing the No Child Left Behind Act: Implications for Rural Schools and Districts 
(Naperville, IL: North Central Regional Educational Laboratory, 2003). 


5. Linda Rosenberg, Megan Davis Christianson, Megan Hague Angus, and Emily Rosenthal, A Focused Look 
at Rural Schools Receiving School Improvement Grants, NCEE 2014-4013 (Washington, DC: Institute of 
Education Sciences, National Center for Education Evaluation and Regional Assistance, April 2014). 


6. Hilda Borko, “Professional Development and Teacher Learning: Mapping the Terrain,” Educational 
Researcher 33, 8 (2004): 3-15. 


7. Rosenberg, Christianson, Angus, and Rosenthal (2014). 
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College Readiness System (ACRS) in eight public schools in the area. The ACRS provides professional 
development to build teaching techniques and strategies that foster critical thinking and strong study 
skills, promotes rigorous course taking and college preparation for all students across a school, and 
provides additional supports for middle-achieving students who aspire to go to college but may not 
be attaining the credentials and skills needed for a seamless transition to postsecondary education. 


Central to the ACRS is a set of AVID “core values,” which promote academic rigor and supports for 
meeting academically high standards in order to prepare for college and to reinforce a college-going 
culture across each school. At the same time, to align curricula, standards, and teaching strategies 
across grade levels and between secondary and postsecondary schools, and to help foster smooth 
transitions from middle school to high school to college, the Collaborative developed and piloted a 
set of “alignment activities” whereby teams of teachers, counselors, and administrators across the 
participating schools meet to make sure they are implementing the ACRS in the same way and to 
collaborate and learn best practices from one another. 


This report presents the key findings from a three-year study, conducted by MDRC, of the imple- 
mentation and outcomes of the ACRS and alignment activities in four middle schools and four high 
schools in rural central Florida. Because the study is nonexperimental, it can identify associations 
between the AVID intervention and any observed outcomes, but it cannot establish that the inter- 
vention is the cause of those outcomes. 


KEY FINDINGS 


e Overall, analyses show that the ACRS was implemented successfully at most schools, suggesting 
fairly high fidelity to the model. In contrast, there was mixed success implementing the align- 
ment activities. 


e There was a positive change in teachers’ reported use of most of the ACRS teaching methodologies, 
in teacher and other school staff members’ reported attitudes and actions related to the AVID core 
values, and in teacher and other school staff members’ reported levels of collaboration within and 
across schools during the first year of implementation. These changes tended to be maintained 
across the three study years. 


e Little difference was found in the reported use of learning skills, engagement in school, and post- 
secondary expectations of the tenth-grade students who were surveyed before full implementation 
and those who were surveyed after three years of exposure to the ACRS. On average, both groups 
had relatively high positive responses on most of the measures related to these outcomes. 


e Students across all eight schools during the third year of implementation were more likely than 
students before implementation to take and earn credits in advanced courses, including honors 
and Advanced Placement courses,® which offer opportunities to attain college credit during high 


8. Advanced Placement courses are college-level classes taught at the high school that prepare students for 
college placement tests that, if passed, can lead to earning college credit. 


ES-2 | Building College Readiness Across Rural Communities 


school. This is important because a key goal of the ACRS and alignment activities was to ensure 
that more students were participating in and succeeding in a more rigorous curriculum in order 
to be better prepared for college. 


e Little difference was found between students in the third year of implementation and those in 
the three years before implementation in other measures of their academic performance (grade 
point average, or GPA, and standardized tests), educational attainment (overall credits earned 
and graduation), and high school persistence (whether they stayed in school through the end of 
the school year). 


THE AVID CENTRAL FLORIDA COLLABORATIVE 


The AVID Central Florida Collaborative is a partnership among the AVID Center, three school dis- 
tricts that border each other in rural central Florida, the local educational consortium, and South 
Florida State College. The AVID Center is a nonprofit organization whose mission is to support 
schools in closing the achievement gap for minority and low-income students. First established in 
1980 in one San Diego high school, the AVID Center currently serves nearly 1.5 million students in 
over 6,100 schools that cover kindergarten through twelfth grade and 45 colleges and universities 
across the country and abroad.° 


In 2012, the Collaborative won an Investing in Innovation (i3) development grant from the U.S. 
Department of Education to build its area’s secondary teaching capacity, strengthen the academic 
rigor of its classes, ensure alignment between grade levels and across secondary and postsecondary 
standards and expectations, and improve transitions between secondary and postsecondary educa- 
tion for students. The i3 grant also supported an evaluation of implementation and outcomes over 
the first three years of the project. The implementation study examines the fidelity of the actual 
implementation to the model design, and examines the factors that helped and hindered successful 
implementation. The outcomes study uses a pre-post nonexperimental method to compare both 
school staff outcomes and student outcomes before implementation with similar outcomes during 
each of the three years of implementation to explore the promise of the system to positively affect 
schools and students. 


Four pairs of schools — with each pair consisting of one middle school (sixth through eighth grades) 
and one high school (ninth through twelfth grades) — are participating in the study, and in each 
case all the students from a single middle school move on to attend the high school with which it is 
paired. South Florida State College, the sole public college in the area, has a satellite campus in each 
of the three counties (DeSoto, Hardee, and Highland) of the participating school districts. 


THE ACRS AND ALIGNMENT ACTIVITIES 


The AVID Central Florida Collaborative focused on two core efforts that were new to the participat- 
ing districts and schools: (1) implementing the ACRS in each of the four pairs of middle schools and 


9. See “AVID Snapshot” and “About AVID” at www.avid.org. 
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high schools, and (2) designing and implementing alignment activities that led to “communities of 
practice” among administrative staff members and teachers from the districts, schools, and college 
to align their efforts to implement the ACRS and to share best practices. 


The ACRS 


The ACRS is focused on building students’ critical thinking, reading, and writing skills and strength- 
ening their study habits and organizational skills. It pursues its goals through its three core compo- 
nents: professional development, an elective class, and school-based “site teams” that work to ensure 
schoolwide implementation of the AVID methods. Central to the ACRS are the aforementioned AVID 
core values and the AVID teaching methodologies, called the WICOR (writing, inquiry, collaboration, 
organization, and reading) model — a set of tools that teachers can use to create a rigorous college 
preparatory environment in their classrooms. The WICOR model is designed to build students’ 
reading strategies, study habits, and critical thinking skills in all subject areas, particularly in the 
core subject areas of English/language arts, math, science, and social studies. Box ES.1 presents more 
detail about the five WICOR domains. 


The ACRS professional development component includes training at a three-day Summer Institute 
held off-site, local training during the school year, and coaching sessions for the AVID elective teach- 
ers and AVID coordinators — teachers or school administrators who ensure ACRS implementation 
across the school and lead the school’s site team. Teachers are trained in the WICOR model and all 
school staff members are trained in the AVID core values. 


Along with supporting the whole school through teacher and staff professional development, the 
system also provides targeted support anchored by the AVID elective class, designed for middle- 
achieving students (those earning mostly Bs and Cs) who have the desire to go to college and the 
willingness to work hard, but whose grades are not generally high enough to get them into more 
advanced courses. During the elective class, students develop their organizational and study skills, 
tackle problems in their school work with the support of trained tutors, and explore college and 
career options. 


Finally, each of the eight schools participating in the study has a site team made up of the school’s 
principal or assistant principal, AVID coordinator, teachers, and other school staff members who 
support the implementation of the AVID elective, address issues affecting student access to and 
success in rigorous courses, and work to ensure schoolwide implementation of AVID methodologies 
through teacher and staff training. 


Alignment Activities 


Education research confirms the need for teacher collaboration across grade levels to ease difficult 
transitions from preschool through postsecondary education and to align content and teaching 
methodologies.'° To enable collaboration and alignment across educational institutions in the 


10. Joan McRobbie, School & College Partnerships: The Missing Link, WestEd Policy Brief (March 2004). 
Website: https://www.wested.org/resources/school-college-partnerships-the-missing-link/. 
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BOX ES.1 
The WICOR Model 


AVID methodologies are used to build students’ reading strategies, study habits, and critical 
thinking skills. The WICOR model strategies represent the AVID teaching methodology, which 
includes a set of tools teachers can use to create a rigorous college preparatory environment in 
their classrooms. 


e Writing includes the AVID notetaking system, an adaptation of the Cornell system, which 
teaches students to take notes in a right-hand column and formulate questions based on the 
notes in the left-hand column,* and other activities to engage students in writing for learning. 


e Inquiry trains students in the inquiry method, which uses levels of questioning — define, 
analyze, and apply — to encourage students to be responsible for their own learning process! 


¢ Collaboration sets up teachers as guides, facilitators, and coaches in a learning community 
and creates collaborative structures for successful group work. 


e¢ Organization is a set of tools for students to organize their study materials and their time to 
maximize learning — for example, the use of binders to organize class work and homework 
across Classes, creating calendars and agendas to keep on task, and using graphic organizers 
and reading logs to structure class work. 


e Reading emphasizes critical reading and uses scaffolding — where teachers first model a 
method or skill and then gradually shift responsibility over the learning process to the students 
— to help students develop their reading skills. 


“Walter Pauk, How to Study in College (Boston: Houghton Mifflin, 2001). 


tArthur L. Costa, Developing Minds: A Resource Book for Teaching Thinking, Third Edition (Alexandria, VA: 
Association for Supervision and Curriculum Development, 2001). 


region, the AVID Central Florida Collaborative developed and piloted four core alignment activi- 
ties especially for this project: four middle school—high school “feeder teams” (one for each middle 
school—high school pair), teacher content collaboratives, a vertical articulation collaborative, and a 
partnership with the local state college. The principals (or assistant principals) and the AVID coor- 
dinators from the two site teams in a middle school—high school pair make up a feeder team, which 
is tasked with ensuring the alignment of the ACRS programming and curricula across grade levels 
as well as facilitating students’ transition from middle school to high school. Each teacher content 
collaborative comprises teachers from a core content area — English/language arts, math, science, 
or social studies — who meet to align college preparation curricula and share best practices for us- 
ing the WICOR model. The vertical articulation collaborative includes school principals, district 
leaders, and representatives from the state college and the AVID Center, who meet to strategize 
about full implementation of the ACRS and alignment activities, align curricular goals, and share 
best practices. Both the teacher content collaborative and the vertical articulation collaborative are 
communities of practice that allow time for professionals (teachers in the content collaboratives and 
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mostly administrators in the vertical articulation collaborative) to share experiences and learn from 
one another. Finally, for the state college partnership, state college instructors and administrators 
receive AVID professional development and participate in the teacher content collaboratives and 
the vertical articulation collaborative. 


THE THEORY OF CHANGE 


The theory of change posits that if implemented as designed, the ACRS and alignment activities would 
lead to more rapid and widespread adoption of the WICOR model and incorporation of AVID core 
values, along with increased collaboration by teachers and other school staff members, as illustrated 
in the logic model in Figure ES.1. Those outcomes, in turn, would lead to improved learning skills, 
heightened engagement in school, and increased motivation to attend college for students across the 
participating middle schools and high schools, resulting in improved academic achievement and, 
ultimately, leading to higher college enrollment and success. 


THE EVALUATION DESIGN 


The evaluation examines the first three years of implementation (the 2013-2014, 2014-2015, and 
2015-2016 school years) and includes an implementation study and an outcomes study. 


Implementation Study 


The goal of the implementation study is to learn whether the ACRS and alignment activities were 
implemented as designed (that is, with fidelity to the model) and what factors may have made imple- 
mentation easier or more difficult. Two primary questions have guided the implementation study: 
(1) To what extent was the system implemented with fidelity to the model? and (2) What factors 
enabled or hindered the implementation of the system? 


Two strands of implementation research were employed to answer these questions: measurement 
of implementation fidelity and qualitative research. For the former, the study team collected yearly 
measures of implementation of each of the key AVID components to understand the fidelity of the 
actual implementation in each year compared with the model. The qualitative research explored 
the challenges and successes in implementation through the collection and synthesis of data from 
interviews and focus groups of key school, district, college, and AVID Center staff members and 
middle school and high school students. 


Outcomes Study 


The outcomes study provides information about how schools and students changed (or did not 
change) during the first three implementation years. The first part of this study focuses on school 
outcomes (shown in the middle column of Figure ES.1), addressing questions about teacher practice, 
expectations for students, and the collaboration of teachers and other school staff members. Research 
questions related to school outcomes include the following: 
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FIGURE ES.1 


Logic Model for the AVID College Readiness System (ACRS) and 


KEY COMPONENTS 


ACRS IN SCHOOLS 


Professional development 
Summer and school-year 
training and coaching for 
teachers and school staff 


AVID elective 

Year-long class to support a 
subset of middle-achieving 
students 


Site team 

Small group of school staff 
leading schoolwide AVID 
implementation 


ALIGNMENT ACTIVITIES 
ACROSS SCHOOLS AND 
DISTRICTS 


Middle school - high school 
feeder teams 


Teacher content 
collaboratives 

English language arts, math, 
science, and social studies 


Vertical articulation 
collaborative (VAC) 

School principals, district 
leaders, and representatives 
from the state college, 

the Heartland Educational 
Consortium, and the AVID 
Center 


State college partnership 


Alignment Activities 


SCHOOL OUTCOMES 


STUDENT OUTCOMES 


Increased knowledge and 
use of the WICOR model by 
all teachers throughout the 
schools? 


Increased use of learning 
skills, including reading and 
study strategies 


Increased student 
Increased knowledge and engagement in school 
incorporation of AVID core 


values throughout the schools Increased expectations and 


awareness of postsecondary 
options among students 


Increased collaboration 
within and between schools 
to strengthen rigor and 
alignment of courses 


Increased enrollment and 
credits earned in Advanced 
Placement, honors, and dual 
enrollment classes 


Improved academic 
performance, educational 
attainment, and high school 
persistence 


Strengthened college 
enrollment and persistence 


NOTES: *The WICOR model includes teaching strategies in writing, inquiry, collaboration, organization, and 
reading. 
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e Do teachers in participating schools report annual increases in the use of WICOR model strate- 
gies in their classrooms? 


e Do teachers, guidance counselors, and principals at participating schools report annual increases 
in knowledge and incorporation of AVID core values throughout their school? Do they report 
increases in collaboration in order to strengthen the rigor and alignment of courses within and 
across participating schools and the college? 


School outcomes were measured using a survey of teachers, guidance counselors, and administrators 
conducted during the spring before implementation began and each spring of the first three years of 
implementation. The survey was administered at all the participating middle schools and high schools. 


The second part of the outcomes study focuses on student behaviors and educational outcomes 
(shown in the rightmost column of Figure ES.1). Research questions related to the student outcomes 
include the following: 


e Does the system offer promise in terms of strengthening all students’ reported use of reading and 
study strategies, engagement in school, and awareness of postsecondary options and expectations 
for postsecondary success? 


e Does the system offer promise in terms of improving the following outcomes: (1) the likelihood 
that all students across the schools will enroll and succeed in more advanced courses (honors, 
Advanced Placement, and dual enrollment); (2) students’ average academic performance (GPA 
and state assessments); (3) students’ overall educational attainment (credits earned, promotion to 
the next grade, and high school graduation); and (4) students’ persistence in school throughout 
the school year? 


Student outcomes were measured using both a student survey and student-level administrative re- 
cords collected from the school districts. The student outcomes findings in this report focus on the 
outcomes of all students in the school, not just students participating in the AVID elective course. 
The goal of the study is to understand the promise of the entire system to positively affect all the 
students across the schools. The study team conducted analyses comparing the years before and 
after implementation began using a pre-post analysis method to measure changes in outcomes over 
time at the participating schools. These analyses are nonexperimental — that is, the study is not 
a randomized controlled experiment, in which the outcomes for two equivalent groups (one that 
receives the intervention and one that does not) are compared to determine the effect. Thus, the 
study identifies whether outcomes changed over time, but not whether implementation caused any 
change in outcomes. 
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STUDY FINDINGS 


Implementation of the ACRS and Alignment Activities 


In general, the ACRS components were implemented successfully, with some components taking a 
little longer than others to fully implement. 


e The professional development sessions were delivered as planned. Participation among teachers 
and staff generally met the goals of the model. 


e The AVID elective was generally implemented with success throughout the three study years, 
although some schools struggled to fully implement the tutorial portion of the elective because 
it was difficult to find and maintain tutors in this rural area. 


e Although every school had a site team in the first year, it was not until the third year that all 
schools were holding regular site team meetings with the appropriate staff members in attendance. 
High turnover of teachers and administrators at some schools may have hindered the successful 
implementation of this component. 


Because the alignment activities were new features designed specifically for this project, it is not 
surprising that they were more difficult to implement and took longer to establish than the ACRS 
components, which the AVID Center had already been using nationwide. As described below, how- 
ever, some student outcomes improved despite the difficulties implementing the alignment activi- 
ties — perhaps driven by the more successful implementation of the ACRS, which was designed to 
improve student outcomes on its own. 


e Middle school and high school feeder teams met sporadically in the first two years. The feeder 
teams were meeting more regularly by the third year, but most of them had still not developed 
an actionable plan by that time. 


e The teacher content collaboratives also struggled to get off the ground, suffering from a lack of 
definition and focus, as well as attendance challenges; it was difficult to find substitute teachers 
to take over content teachers’ classrooms during the full-day meetings. 


e District and school administrators met and created a productive community of practice through 
the vertical articulation collaborative. 


e The college partnership was not successful in that the expected number of college faculty and 
administrators did not always attend the teacher content collaborative and vertical articulation 
collaborative meetings. However, college faculty participated in more professional development 
sessions than were originally planned. 


The ACRS and alignment activities benefited from the strong support of the AVID systems coach 
— an AVID Center employee who guided all the program activities within and across schools — as 
well as the active engagement of school administrators and the robust community of practice created 
by the collaboration of district and school administrators and some teachers. 
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Adoption of the WICOR Model, AVID Core Values, and Collaborative 
Approach 


The growth of the ACRS across the schools may have contributed to the changes found in teachers’ 
reported use of most of the WICOR model strategies — many of which reflect good teaching prac- 
tices that could have been learned in other training sessions before the AVID study began. Teachers 
reported using these types of strategies more during the first year of implementation than they had 
before implementation, and that level of reported use in Year 1 held steady for the subsequent two 
years. Teachers, guidance counselors, and administrators at the schools also reported a higher level 
of conviction about the importance and feasibility of instilling academic rigor and creating a college- 
going culture across the school — which make up the AVID core values — during the implementation 
years than before implementation. 


Student Behaviors, Attitudes, and Academics 


The reported use of learning skills, engagement in school, awareness of postsecondary opportunities, 
and planned postsecondary degree attainment among tenth-graders before implementation compared 
with another group of tenth-graders after three years of implementation showed little difference 
between the two groups, but both groups had relatively positive responses on most of these measures. 
Students increased the number of credits earned in advanced courses, with 8 percent more students 
passing at least one advanced course in the third year of implementation than passed in the years 
before implementation. Other student outcomes, including course grades, state assessment scores, 
educational attainment, and school persistence, did not change substantially on average. 


CONCLUSIONS 


The findings that teachers increased their use of best-practice instructional methodologies and 
that staff members across the schools strengthened their conviction that all students can succeed 
in college both point to an important culture shift at the schools, moving toward a stronger belief 
that more students should take and can succeed in rigorous course work. This change in attitude 
may have helped lead to the positive finding that students were taking and succeeding in more ad- 
vanced courses during the first three implementation years than was observed in the schools before 
implementation began. 


Research suggests that content area teachers across grades and across secondary and postsecondary 
institutions coming together in communities of practice — to ensure that they share expectations of 
students and to help pave the way for smooth transitions between grade levels — is key to students’ 
postsecondary education success." Establishing this type of community across schools and colleges 
is not easy, and the difficulties can be exacerbated by the sheer physical distance between schools 
in rural areas. While the Collaborative partners struggled with implementing the teacher content 
collaboratives, lessons were learned from the process about the need to include school and district 
leaders in meaningful ways and about the importance of setting meetings at times when teachers 
can realistically attend. 


11. McRobbie (2004). 
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One positive finding that might portend future success in this area was the embrace of AVID teach- 
ing methodologies by the local state college. The use of shared language and a set of methodologies 
among secondary and postsecondary educators could allow for more effective future dialogues. 
Moreover, according to the theory of change, the consistency — or alignment — of methods in the 
secondary and postsecondary education experiences of students could eventually lead to longer-term 
college success by easing the transition into postsecondary education for these students. 
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Introduction 


ttaining some postsecondary education or a postsecondary credential is associated with 
achieving better employment outcomes and earning a living wage in the current United 
States labor market. According to a 2016 report from the Center on Education and the 


Workforce at Georgetown University, a majority of the 11.6 million jobs that have been 
added to the economy since the Great Recession (2007 to 2009) have gone to workers with at least 
some college education.' However, less than half (46 percent) of 25-29-year-olds across the United 
States had earned at minimum an associate’s degree in 2016. One obstacle to degree attainment 
may be that many high school graduates arrive at college unprepared for college-level course work. 
According to the National Center for Education Statistics, only 37 percent of high school seniors 
scored at or above proficient in reading and only 25 percent of seniors scored at or above proficient 
in math on the National Assessment of Educational Progress in 2015.° 


Rural secondary schools face many unique challenges in preparing students for college and inspiring 
them to enroll. Rural schools often encompass large geographic areas with lower population density 
and a more limited tax base than is found in urban and suburban settings, forcing rural districts 
to stretch their resources further.* These compromises can affect the quality of instruction. For 
instance, teachers in rural areas are often asked to cover multiple and different classes or subjects, 
so they need more time to prepare and they have less time for professional collaboration,° which is 
important for strengthening teaching and learning skills.® Given their distance from urban centers, 
many rural schools face challenges in recruiting and retaining qualified teaching staff and in offer- 
ing the professional development and training needed to build leadership across the school.’ Given 
these challenges, some rural school districts in low-income communities are looking for ways to 
build their secondary teaching capacity, strengthen the academic rigor of their classes, ensure align- 
ment between grade levels and across secondary and postsecondary standards and expectations, 


Carnevale, Jayasundera, and Gulish (2016). 

McFarland et al. (2017). 

National Center for Education Statistics (2015). 

Reeves (2003). 

Rosenberg, Christianson, Angus, and Rosenthal (2014). 
Borko (2004). 

Rosenberg, Christianson, Angus, and Rosenthal (2014). 
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and build better transitions between middle schools and high schools and between secondary and 
postsecondary education for their students. 


One such effort is the AVID (Advancement Via Individual Determination) Central Florida Collaborative 
(abbreviated as “Central Florida” throughout this report), a partnership among the AVID Center 
(described below), three school districts in rural central Florida, the local educational consortium, 
and the local state college focused on building the college preparedness of the students in middle 
school and high school across the area. In 2012, these partners won an Investing in Innovation (i3) 
grant from the U.S. Department of Education to develop a collaborative model that supports a broad 
group of students attending secondary and postsecondary institutions in a rural area. 


To reach its objective, the AVID Center and its Central Florida partners implemented the AVID 
College Readiness System, referred to as the ACRS throughout this report. The ACRS builds on the 
longstanding and popular AVID elective program (described below), which offers substantial sup- 
ports to a set of students within a school by working to ensure that strong study and organizational 
strategies are being taught to all students and that college readiness is promoted throughout the 
school. The ACRS provides professional development to build teaching techniques and strategies 
that foster critical thinking and strong study skills, promotes rigorous course taking and college 
preparation for all students across a school, and provides additional supports for middle-achieving 
students (that is, students receiving mostly Bs and Cs in their classes) who aspire to go to college 
but may not be attaining the credentials and skills needed for a seamless transition to postsecond- 
ary education. At the same time, the partners wanted to help teachers align curricula, standards, 
and instructional strategies across grade levels and between secondary and postsecondary schools, 
and to help foster smooth transitions from middle school to high school to college. To that end, 
they developed and piloted a set of alignment activities whereby teams of teachers, counselors, and 
administrators across the participating middle schools, high schools, and the college meet to align 
their implementation of the ACRS and to collaborate and learn best practices from one another. 


THE AVID CENTRAL COLLABORATIVE PARTNERS 


As explained above, the AVID Central Florida Collaborative partners comprised the AVID Center, 
three central Florida school districts, and a local college. The three districts are part of the Heartland 
Educational Consortium, a regional nonprofit educational service agency that helped administer 
the grant. 


The Avid Center. The AVID Center is a nonprofit organization whose mission is to support schools 
in closing the achievement gap for minority and low-income students. First established in 1980 in 
one San Diego high school, as of 2017 the AVID Center was serving nearly 1.5 million students in 
more than 6,100 schools comprising kindergarten through twelfth grade, and 40 colleges and uni- 
versities internationally.? The AVID Center worked closely with the schools and districts to plan and 
implement the ACRS and alignment activities and provided schools with professional development, 
support, and materials along the way. 


8. AVID (2017a, 2017b). 
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Central Florida School Districts. The three participating school districts border each other in a rural 
area of central Florida approximately 80 miles southeast of Tampa. The School District of DeSoto 
County and Hardee County School District each include one high school and one middle school 
participating in the project, and the School Board of Highlands County has two middle schools and 
two high schools participating in the project. These eight schools make up four middle school-high 
school pairs. In each case, the students from a single middle school (sixth through eighth grades) 
move on to attend the high school (ninth through twelfth grades) with which it is paired. Staff in 
each of the three districts helped oversee the implementation of the system, and teachers and staff at 
all the schools participated in the professional development and alignment activities and in imple- 
menting the system across their school. All the schools in the study were new to the ACRS and had 
not worked with the AVID Center before. 


South Florida State College. The sole public college in the area, South Florida State has a satellite 
campus in each county where the participating school districts are located. College instructors were 
also trained in AVID instructional strategies and participate in alignment activities along with 
teachers and administrators from the middle schools and high schools. The college was also new to 
the ACRS and to working with the AVID Center. 


THE EVALUATION 


This report presents both implementation and outcome findings stemming from a three-year 
evaluation, conducted by MDRC, of the Central Florida Collaborative partners’ efforts. MDRC has 
worked with the partners throughout the program implementation, conducting data collection and 
analyses and providing the AVID Center and other partners with formative feedback along the way. 
The implementation study examines the fidelity of the actual implementation of the system to the 
model design, and the enablers of and obstacles to successful implementation, across all eight par- 
ticipating schools and the state college. Program data and qualitative data are collected from staff 
and students of two middle school-high school pairs in two different districts, the administrators 
of those two districts, the state college staff, and key AVID Center staff. The outcomes study uses 
a pre-post nonexperimental method to compare both school staff outcomes and student outcomes 
before the implementation of the ACRS and alignment activities with similar outcomes during each 
of the three years of implementation (the 2013-2014, 2014-2015, and 2015-2016 school years); the 
purpose was to explore the promise of the system to positively affect school and student outcomes. 
Nonexperimental methods do not capture whether the program implementation caused any changes 
in school or student outcomes — the analyses look at associations only. 


The rest of this chapter describes the ACRS and the alignment activities in more detail, explains the 
theory of change, and describes the schools and districts participating in the study. 


THE ACRS AND ALIGNMENT ACTIVITIES 


The AVID Center and its Central Florida Collaborative partners focused on two core efforts that 
were both new to the participating districts and schools: (1) implementing the ACRS in each of the 
four middle school-high school pairings, and (2) designing and implementing alignment activities 
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that brought together staff and teachers from each of the districts and schools, plus the college, to 
align their efforts to implement the ACRS and create opportunities for staff members to share best 
practices. 


The AVID College Readiness System 


Designed by the AVID Center and implemented to varying degrees in hundreds of secondary schools 
across the country, the ACRS includes resources and professional development to strengthen and 
align teaching strategies across the school. These strategies focus on building students’ critical 
thinking, reading, and writing skills, and strengthening their study habits and organizational skills. 
The AVID Center also works with participating districts and schools to help them align their im- 
provement initiatives, emphasizing college readiness across all schools and for all students. Along 
with supporting the whole school through teacher and staff professional development, the system 
provides a targeted service, an elective class, designed for middle-achieving students who want to 
go to college and are willing to work hard, but whose grades are not generally high enough to get 
into more advanced courses. Many of the students targeted for participation in the AVID elective 
class come from backgrounds that have traditionally been underrepresented in higher education, 
including low-income and minority students. Through participation in the AVID elective, these 
students are pushed to take more advanced courses and are offered additional academic and social 
support to help them succeed in these courses. 


Central to the ACRS is the WICOR (writing, inquiry, collaboration, organization, and reading) 
model, which includes a set of teaching and learning methodologies that educators can use to guide 
students in comprehending concepts and articulating ideas at increasingly complex levels, and the 
philosophy — represented by AVID’s “core values” (described in Box 1.1) — that all students should 
be held to high academic standards and provided the academic and social support to meet those 
standards. The WICOR model (described in Box 1.2) is designed to add rigor and engagement to 
classroom instruction and students’ academic work in all subject areas — in particular, in the core 


subject areas of English/language arts, math, science, and social studies. Furthermore, the WICOR 


BOX 1.1 
AVID Core Values 


The AVID Center’s core philosophy is that all students should be held accountable to high 
academic standards and given the academic and social support they need to meet those 
standards. The core values that AVID works to spread across a school are that all students 
should receive a rigorous academic experience and can and should be prepared for college. 
Incorporating these core values into the school culture is important to the effectiveness of 
AVID because it ensures that schools will open up rigorous courses such as honors, Advanced 
Placement, and dual enrollment classes to more students, while offering those students the 
support they need to succeed in those courses. 
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model reflects and promotes AVID’s core values, which reinforce a college-going culture across 
each school. 


The ACRS comprises three main components: the schoolwide professional development, the AVID 
elective, and the site team. Each one has an active role in promoting AVID’s WICOR model and 
core values. Several staff members across the AVID Center, districts, and schools play key roles in 
implementing the ACRS. Box 1.3 describes each of these important players. The AVID Center works 
with district and school staff to align all school improvement and district initiatives in one system. 
Each school goes through an annual certification process called Certification Self-Study (CSS), which 
measures the school’s fidelity to the model in implementing all the components, scores the school 
on a continuum, and identifies areas for improvement during the next year. All the schools in the 


BOX 1.2 
The WICOR Model 


AVID methodologies are used to build students’ reading strategies, study habits, and critical 
thinking skills. The WICOR model represents the AVID teaching methodologies, which include 
a set of tools teachers can use to create a rigorous college preparatory environment in their 
classrooms. 


¢ Writing includes the AVID note-taking system, an adaptation of the Cornell system, which 
teaches students to take notes in a right-hand column and formulate questions based on 
the notes in the left-hand column,* and other activities to engage students in writing for 
learning. 


¢ Inquiry trains students in the inquiry method, which uses levels of questioning — define, 
analyze, and apply — to encourage students to be responsible for their own learning 
process.t 


¢ Collaboration sets up teachers as guides, facilitators, and coaches in a learning community 
and creates collaborative structures for successful group work among students. 


¢ Organization is a set of tools for students to organize their study materials and their time 
to maximize learning — for example, the use of binders to organize class work and 
homework across classes, creating calendars and agendas to keep on task, and using 
graphic organizers and reading logs to structure class work. 


¢ Reading emphasizes critical reading and uses scaffolding — where teachers first model a 
method or skill and then gradually shift responsibility for the learning process to the 
students — to help students develop their reading skills. 


*Pauk (2001). 
tCosta (2001). 
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BOX 1.3 


Key Roles in the ACRS and Alignment 
Activities Implementation 


The AVID project systems coach is a full-time AVID Center employee tasked with supporting 
the implementation of the AVID College Readiness System (ACRS) and the alignment activities 
across all the schools and the college, and providing training and coaching to teachers and staff. 


The AVID district director is a local school district administrator, trained in the WICOR model 
(writing, inquiry, collaboration, organization, and reading) and the AVID core values, who oversees 
and manages the program implementation across the schools in the district. 


The AVID coordinator is a teacher or school administrator who ensures ACRS implementation 
across the school and leads the school’s site team. 


The AVID elective teacher is assigned to the AVID elective class and, ideally, models and 
promotes WICOR model learning methodologies and core values throughout the school. 


Guidance counselors are school staff members whose duties may vary from school to school 
but who serve key functions within an AVID school, including serving on the AVID site team, 
assisting with recruitment and selection of students for the AVID elective, and ensuring access to 
rigorous courses for AVID students. 


Core teachers cover English/language arts, math, science, and social studies; they are the key 
implementers of the WICOR model within the school. 


project were new to the ACRS and the system required a shift in philosophy and mindset for many 
of the staff members and teachers. 


Professional Development 


The main format for bringing the WICOR model to all students and for promoting the AVID core 
values throughout the school is through professional development offered to school administra- 
tors, guidance counselors, and teachers. The AVID Center offers a three-day Summer Institute that 
trains groups of teachers and staff from schools throughout each participating region. Each Institute 
includes sessions for people who are new to the program as well as for those with varying levels of 
experience and knowledge of the program so staff can keep building their skills. Two-day sessions, 
called “Path to Schoolwide,” are also offered locally during the school year. The sessions offer train- 
ing on specific WICOR model methodologies for teachers across the school, as well as providing 
additional support to AVID site teams (described below) and school leadership for spreading the 
WICOR model and AVID core values across the school. For this study, the goal was that 60 percent 
of all teachers at the middle schools and high schools would participate in professional development 
during the grant period. 
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The AVID Elective 


The AVID elective is a year-long course, taught five days a week, for middle school and high school 
students who are interested in going to college but who require additional support to be prepared for 
college by the time they graduate from high school. Before the start of the school year, school staff 
members identify and recruit a set of students who could benefit from the AVID elective; students 
who are selected to enroll must sign a formal letter of commitment to the program, acknowledging 
that they understand the course will require consistent attendance and effort. 


The AVID elective gives students a grounding in study, organizational, critical thinking, and read- 
ing skills. During two of the five AVID elective classes per week, students participate in small group 
tutorials led by a trained college student. These tutorials are meant not only to help students with 
problems they have with their course work, but also to teach them how to identify and summarize 
a problem and how to engage in a dialogue with peers about solutions. During the other three days, 
the AVID elective teacher works with students to build their competencies in the WICOR model 
methodologies and engages students in activities that help them explore and plan for college and 
careers. (See Box 1.4 for a more detailed description of an AVID elective class.) 


Students who take the AVID elective are strongly encouraged to take at least one advanced class 
per semester, and the schools agree to allow these students to take these rigorous classes. Advanced 
classes include honors-level courses, dual enrollment courses (which can be taught at the high school 
or on the college campus and allow students to earn credits from both the high school and the local 
college simultaneously), and Advanced Placement courses (which are college-level classes taught at 
the high school that prepare students for college placement tests that, if passed, can lead to earning 
college credit). Students who enrolled in the AVID elective represent about 10 to 15 percent of the 
total student population across each school. 


Extra professional development support, in the form of group and individual coaching sessions, is 
also provided to AVID elective teachers and AVID coordinators by the AVID systems coach. 


Site Teams 


The site team is a small group of teachers and other school staff members who work together to ensure 
that the AVID elective is fully implemented and who promote the schoolwide use of the WICOR 
model and incorporation of AVID core values. The site team usually includes an AVID coordinator, 
the AVID elective teacher, a guidance counselor, an administrator, and a few other teachers. The 
site team, with the AVID coordinator taking a lead role, actively supports the implementation of 
the AVID elective, helps identify and recruit new AVID elective students at the beginning of each 
school year, and provides training and modeling to other teachers, staff, and tutors at the school 
to advance the schoolwide components of AVID. Site team responsibilities include developing and 
implementing an annual plan for achieving ACRS goals, coordinating “communities of practice” 
(described below) among teachers at the school, disseminating information about AVID to parents 
and the community at large, and fundraising for AVID elective activities such as college field trips. 
The site team also meets at least monthly to engage in problem solving related to issues with student 
access to and success in advanced courses. 
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BOX 1.4 
What Happens in a Well-Implemented AVID Elective Class? 


Scanning the room of an AVID elective class, you will notice that the walls are filled with 
encouraging messages, college pennants, and charts tracking students’ progress in various ways 
(such as their high school attendance and their ability to keep their school binders organized and 
get their course work finished on time). Student-made diagrams explain various reading, writing, 
and study strategies. Each one provides explicit definitions related to the strategy and examples 
of how and when to use it. The AVID elective class is designed to make explicit to students all the 
ways that they can organize their time and all the ways that they can interact with the curriculum to 
master the material and earn good grades. During the course of the year, an AVID elective teacher 
will cover (or reinforce, or have students practice or apply) dozens of the WICOR model (writing, 
inquiry, collaboration, organization, and reading) methodologies. 


AVID students also participate in an in-class tutorial during two of the AVID elective classes per 
week. The tutorial is designed to allow students to practice advocating for themselves, seeking 
support, and learning through collaboration so they are well versed in those practices before 
starting college. The tutorial, in which students meet in small groups with a college-age tutor, 
follows a rigorous protocol that requires every student to prepare and participate. To prepare, 
students select a problem or concept from their homework or classwork with which they are 
struggling. They copy the problem or concept, summarize what they have tried to do or what they 
know about it, and articulate their remaining question(s) or confusion. As the students take turns 
presenting their problems during the tutorial, the rest of the group practices well-defined roles 
and responses. For example, rather than telling their fellow students how to solve the problem, 
students in a well-functioning tutorial will ask probing questions and make comparisons or loose 
suggestions. A primary goal behind the tutorial is to break students of the habit of expecting 
teachers and friends to help them with their academic challenges by simply providing answers. 


In addition, AVID elective students participate in in-depth college and career awareness activities 
in middle school and college- and career-readiness activities in high school. These activities 
include research on actual postsecondary options, full exposure to financial aid options, and tours 
of local colleges and employers. Often this work is done in groups and is self-directed, with the 
teacher acting as a guide and offering one-on-one and small group support. 


Finally, AVID elective students have fun! Tasked with serving as a source of pride and inspiration at 
their school, the AVID elective class will often take on community service projects, fundraisers, and 
schoolwide events to promote school spirit. 


To ensure that the three ACRS components were implemented successfully during the years of the 
grant, the AVID Center and its Central Florida partners designed a plan to accelerate the implementa- 
tion of the three components compared with the usual rollout of the ACRS in a school. Professional 
development was offered to many staff members over the course of the three years. Fifteen staff 
members from each school were invited to the first Summer Institute and eight staff members were 
invited in the two follow-up years. At least 10 staff members were invited to the Path to Schoolwide 
trainings each year. The AVID Center also provided a full-time AVID systems coach to work exclu- 
sively with the districts that were participating in the grant activities. 
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A plan was created for rolling out the ACRS in specific grade levels each year to make sure that all 
grade levels had an AVID elective and were fully implementing the ACRS by the third year. Table 1.1 
shows which grade levels were implementing the ACRS during each year of the study. During the first 
school year (2013-2014), the ACRS was implemented in the seventh and eighth grades at the middle 
schools and in the ninth and tenth grades at the high schools. During the second year (2014-2015), 
the program was implemented in all middle school grades (sixth through eighth) and ninth, tenth, 
and eleventh grades at the high schools, and during the final year of the study (2015-2016), the ACRS 
was implemented in all grades in both the middle schools and the high schools. 


TABLE 1.1 
Grade Levels Supported by the AVID College Readiness 
System (ACRS), by Implementation Year 


ACRS Implemented 


Year 1 Year 2 Year 3 

School Type Grade (2013-2014) (2014-2015) (2015-2016) 
Middle schools 6th 

7th oa Sd Sd 

8th oa ¢ ¢ 
High schools 9th . . 

10th . 4 od ¢ 

11th ° 

12th ° 


Alignment Activities 


Although middle schools, high schools, and local colleges may in fact serve the same students, rarely 
do these schools systemically collaborate to ensure that, from kindergarten through twelfth grade, 
their students will receive rigorous instruction that prepares them for college.? Education research 
confirms the need for collaboration across grade levels in addressing transitions from preschool 
through postsecondary education, aligning college-readiness expectations, and ensuring similarities 
in preparation for teachers across grade levels.'° In an attempt to enable collaboration and align- 
ment across educational institutions in the region, the AVID Center and its Central Florida partners 
developed, piloted, and revised four core alignment activities, or elements: the middle school-high 
school feeder teams, the teacher content collaboratives, the vertical articulation collaborative, and 
the state college partnership. The AVID systems coach was responsible for leading the collaborative 
processes and supporting and guiding the various teams in their planning. At the beginning of the 
grant period, the ACRS was already being implemented in many other schools across the country; 
however, the alignment activities were new to the AVID Center and to the participating schools and 
districts in the study. The alignment activities were designed and piloted especially for this project. 
For this reason, these components were subject to more trial and error than the ACRS and changed 


9. Kirst and Venezia (2001). 
10. McRobbie (2004). 
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from the original design during their rollout over the three years of the study. The original design 
for each of the four components is described below. Chapter 3 discusses the evolution of these pro- 
gram components. 


Middle School-High School Feeder Teams 


Each middle school-high school pair was represented by a “feeder team” that was made up of the 
principal (or assistant principal) and AVID coordinator from each site team in the pair. Thus, there 
were four feeder teams, each representing one of the four middle school-high school pairs. The 
partners planned for the four feeder teams to meet twice per year to align programming for stu- 
dents across grade levels and schools. For instance, the teams planned which AVID tools and skills 
(for example, interactive notebooks, note taking, critical reading, and so on) would be taught and 
reinforced in which grades. The plan was that each feeder team would create a written plan each 
year with specific action items to support the use of the WICOR model and implementation of the 
AVID elective. 


Teacher Content Collaboratives 


Content-area teachers in rural schools often find themselves isolated and geographically distant 
from other teachers in their subject area. Teacher content collaboratives were developed to create 
communities of practice where teachers can learn from each other and can connect college-readiness 
curricula across middle school, high school, and college. A collaborative was designated for each 
core content area (English/language arts, math, science, and social studies). The original plan for 
these collaboratives was that three content teachers from each of the participating middle schools 
and high schools would meet for a full day, twice per year, to share best practices in implementing 
the WICOR model and to align the college preparation curricula across schools. 


Vertical Articulation Collaborative 


It was planned that the vertical articulation collaborative (VAC), composed of school principals 
from all eight schools and a teacher from some schools, district leaders from the three districts, 
representatives from the state college, and a representative from the consortium of rural school dis- 
tricts, would meet three times per year. The VAC was intended to strategize full implementation of 
the ACRS across the participating districts, align curricular goals, and observe and share WICOR 
best practices. VAC members were also invited to participate in “classroom walk-throughs” held at 
different schools to observe the WICOR model being implemented in core subject classes. 


State College Partnership 


The additional partnership of the local state college allows for alignment of teaching strategies and 
curricula across secondary and postsecondary schools. It was planned that 20 college staff members 
would participate in AVID professional development each year, either through participation in the 
Summer Institute or attendance at a training session at the college. In addition, two state college 
instructors in each core subject area were invited to participate in each of the teacher content col- 
laboratives to help align curricula and instructional strategies across secondary, dual enrollment, 
and postsecondary courses in each core subject area. 
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THEORY OF CHANGE 


The theory of change predicts that if implemented as designed, the ACRS and alignment activities 
would lead to widespread adoption of the WICOR model and incorporation of AVID core values, 
along with increased communication and collaboration by teachers and other school staff. Those 
outcomes, in turn, would lead to improved learning skills, heightened engagement with school, and 
increased motivation to attend college for students across the middle schools and high schools, re- 
sulting in improved academic achievement and, ultimately, higher college enrollment and success. 


Figure 1.1 displays this theory of change in detail. The key components of the ACRS and alignment 
activities are represented in the left-hand column. These program components are hypothesized to 
lead to the key school outcomes displayed in the middle column. They include the increased knowl- 
edge and use of the WICOR models during the AVID elective and across all classes at the school; the 
incorporation of the AVID core values, which hold all students to high academic standards, support 
them so they can succeed, and incorporate the belief that all students can be prepared for college; 
and increased collaboration within and across schools by teachers and other school staff, leading 
to a more cohesive, rigorous, and supported school experience for all students. These school-level 
goals are hypothesized to lead to several positive student outcomes, displayed in the far-right col- 
umn of Figure 1.1. The first set of outcomes focuses on students’ increased use of learning strategies, 
including reading and study strategies, their increased engagement and interest in school, and their 
increased awareness of postsecondary educational options and plans for college. Via these first three 
outcomes, the implementation of the program is also hypothesized to lead to students enrolling and 
succeeding in more honors, dual enrollment, and Advanced Placement courses, as well as improved 
academic performance (measured by a student’s grade point average and success on state standard- 
ized tests), educational attainment (credits earned, promotion to the next grade level at the end of 
the school year, and high school graduation), and persistence in high school. The ultimate goal is to 
increase students’ college enrollment and success. 


PARTICIPATING DISTRICTS AND SCHOOLS 


The AVID Center and its Central Florida partners in this study were particularly concerned about 
addressing the specific challenges they face in a rural community with many low-income students 
and families. This section provides more detailed information on the community, the students, and 
the college. 


The Community 


The rural counties where the three participating school districts are located make up a large geo- 
graphic area with low population density. Population estimates from 2016 show that each of the two 
smaller counties — DeSoto and Hardee — has fewer than 36,000 occupants, while Highlands County 
has approximately 100,000 people. Based on available geographic data from 2010, Florida’s average 
population per square mile (350.6) is more than three times as great as the average for Highlands 
County (97.2) and over six times as great as DeSoto County (54.7) and eight times as great as Hardee 
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FIGURE 1.1 


Logic Model for the AVID College Readiness System (ACRS) and 


KEY COMPONENTS 


ACRS IN SCHOOLS 


Professional development 
Summer and school-year 
training and coaching for 
teachers and school staff 


AVID elective 

Year-long class to support a 
subset of middle-achieving 
students 


Site team 

Small group of school staff 
leading schoolwide AVID 
implementation 


ALIGNMENT ACTIVITIES 
ACROSS SCHOOLS AND 
DISTRICTS 


Middle school - high school 
feeder teams 


Teacher content 
collaboratives 

English language arts, math, 
science, and social studies 


Vertical articulation 
collaborative (VAC) 

School principals, district 
leaders, and representatives 
from the state college, 

the Heartland Educational 
Consortium, and the AVID 
Center 


State college partnership 


Alignment Activities 


SCHOOL OUTCOMES 


STUDENT OUTCOMES 


Increased knowledge and 
use of the WICOR model by 
all teachers throughout the 
schools? 


Increased use of learning 
skills, including reading and 
study strategies 


Increased student 
Increased knowledge and engagement in school 
incorporation of AVID core 


values throughout the schools Increased expectations and 


awareness of postsecondary 
options among students 


Increased collaboration 
within and between schools 
to strengthen rigor and 
alignment of courses 


Increased enrollment and 
credits earned in Advanced 
Placement, honors, and dual 
enrollment classes 


Improved academic 
performance, educational 
attainment, and high school 
persistence 


Strengthened college 
enrollment and persistence 


NOTES: “The WICOR model includes teaching strategies in writing, inquiry, collaboration, organization, and 
reading. 


12 | Building College Readiness Across Rural Communities 


County (43.5)."' At the start of the project, each of the school districts was an eligible rural local edu- 
cation agency (LEA) under the Rural and Low-Income (RLIS) grant program, which provides rural 


districts with federal financial assistance for initiatives aimed at improving student achievement.” 


The three counties also have lower average incomes, and lower rates of educational attainment, than 
the statewide averages. From 2012 through 2016, the median annual household income for the state 
was approximately $48,900, compared with approximately $36,000 in each of the three study coun- 
ties. Across those same years, the percentage of the population age 16 years and older in the civilian 
labor force was 43.1 to 51.5 percent across the three counties, while across the state, employment for 
this group was at 58.5 percent. Between 2012 and 2016, the percentage of bachelor’s degree holders 
age 25 years and above was 9.6 to 16.5 percent across the three counties and 27.9 percent statewide.'® 


The Schools 


All the participating secondary institutions are Title 1, a status created by the federal Elementary and 
Secondary Education Act indicating schools with high numbers or high percentages of children from 
low-income families; more than 60 percent of students across these schools are eligible for free or 
reduced-price lunch, which is a proxy for children from low-income families. Over half the students 
at the participating schools are Hispanic or black, but the percentage of English language learners 
(that is, children whose native language is not English) is low, at about 2 percent. Approximately 
48 percent of students meet proficiency standards on sixth grade standardized tests for math, and 
approximately 53 percent of students met proficiency standards for English.'* Before the project 
began, many of the participating schools were low performing according to their school report cards 
released by the state’s education department. One school had a failing report card, three had earned 
Ds, and the other four had Cs in the year before the start of the program.”® 


The College 


In 2012, the local community college changed its status to a state college and started offering some 
four-year degrees along with associate’s degrees in arts and sciences. The state college offers dual 
enrollment opportunities for high school students, including all four high schools participating 
in this project. The average student age at the college is 23, with 64 percent of students attending 
part time.’ The graduation rate is 39.5 percent for full-time, first-time, degree/certificate-seeking 
undergraduates within 150 percent of normal time to completion (for example, within six years for 
a four-year bachelor’s degree program).'” 


11. U.S. Census Bureau (2016). 

12. U.S. Department of Education (2017). 

13. U.S. Census Bureau (2016). 

14. See Appendix Table C.1 for more information about student characteristics. 
15. Florida Department of Education (2014). 

16. South Florida State College (2017). 

17. South Florida State College (2016). 
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STRUCTURE OF THIS REPORT 


This first chapter introduced the partnership and grant, described the ACRS and alignment activities, 
discussed the theory of change for this project, and described the partner districts, communities, 
schools, and the participating college. Chapter 2 describes the study design. Chapter 3 describes the 
findings from the implementation research. Chapter 4 compares the use of the AVID methodologies 
by school staff during the first three years of implementation with their use during the year before 
implementation. Chapter 5 discusses student outcomes pertaining to student behavior and engage- 
ment, postsecondary expectations, and academic performance and persistence in school. Finally, 
Chapter 6 sums up the findings across the different parts of the study and offers some concluding 
thoughts about those findings. 
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AVID Study Design 


ver the three school years included in the AVID study (2013-2014, 2014-2015, and 

2015-2016), the study team collected yearly measures of implementation to learn whether 

the program was actually implemented with fidelity to the model and goals; explored 

challenges and successes in implementation by collecting and synthesizing qualitative 
data; surveyed teachers, principals, and guidance counselors to measure how the AVID College 
Readiness System (ACRS) and alignment activities affected teacher practice and the attitudes of 
school staff regarding students’ college readiness; surveyed students to capture any changes in their 
activities, attitudes, and perspectives; and collected student-level administrative records data from 
the districts to measure changes in student academic outcomes. 


The focus of the grant that supported the AVID study was on designing and developing an effective 
system as opposed to measuring the impacts of that system. Accordingly, the study focuses on the 
successful development and implementation of the ACRS and alignment activities. The analyses of 
outcomes in this study are all nonexperimental and can only show whether the ACRS and alignment 
activities are correlated with any school or student outcomes. They do not capture whether the imple- 
mentation of the ACRS or the alignment activities caused any changes in school or student outcomes. 


This chapter describes the study design, including the research questions, data collection, measure- 
ment and analyses, and the study sample for each component of the study. It also explains how the 
program was rolled out in specific grade levels over the three-year follow-up period. 


IMPLEMENTATION OF THE ACRS AND ALIGNMENT ACTIVITIES 


The implementation study investigates what it takes to develop and implement the ACRS and align- 
ment activities, examines the fidelity of implementation to the project design, documents experiences 
of both student and staff participants, and describes contextual factors that may have affected project 
implementation. The goal of this part of the study is to generate a picture of the implementation 
process and resources, as well as to learn what may be needed for further development or future 
replication. The primary implementation questions guiding this part of the study are: 


e To what extent was the program implemented with fidelity to the model? 


e What factors enabled or hindered the implementation of the program? 
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Two strands of implementation research were employed to answer these questions: measurement of 
implementation fidelity and qualitative research. The study sample includes all four pairs of middle 
schools and high schools as well as the local state college. 


Measurement of the Fidelity of Implementation 


The fidelity measurement looks at the implementation of each of the three ACRS components 
(professional development, the AVID elective, and the site teams) and the four alignment activity 
components (the feeder teams, the teacher content collaboratives, the vertical articulation collab- 
orative, and the state college partnership), discussed in Chapter 1. The study team worked closely 
with the AVID Center to identify the key components and the best method to measure the fidelity 
of implementation of each component. 


The ACRS 


The three components that make up the ACRS were measured at the school level. Scores for the 
professional development component are based on the participation rate compared with the partici- 
pation goals for each of the three types of professional development offered (Summer Institute, Path 
to Schoolwide, and coaching of the AVID elective teachers). Scores for fidelity of implementation of 
the AVID elective are based on six elements of the AVID elective component: (1) the elective class 
being offered at each grade level, (2) meeting student recruitment and enrollment goals,’ (3) teach- 
ing organizational skills during the class, (4) teaching reading and writing strategies, (5) teaching 
inquiry and collaboration skills, and (6) providing support from trained tutors as part of the AVID 
elective. Scores for the site teams component are based on one element: the existence of an active 
interdisciplinary AVID site team collaborating on issues of student access to and success in rigor- 
ous college preparatory courses. 


The six AVID elective elements and one site team element were measured using data from the AVID 
Center’s certification documents, rubrics developed to help schools understand how well they are 
implementing pieces of the ACRS. Each school received a score of low/no implementation, partial 
implementation, or full implementation for each of the six elements listed above. The scores for 
the elements under each component were tallied to create an implementation score for each school, 
reflecting the extent to which the school implemented the component as designed during each year 
of the study. 


Alignment Activities 


For the four alignment activities across schools and districts, the level of measurement depended 
on the level of the collaboration. Each middle school-high school feeder team received an imple- 
mentation score each year. Implementation scoring for feeder teams was based on two elements: 


1. These goals include the following: (1) 100 percent of the AVID elective students must meet the school 
district’s defined selection criteria classifying them as students in the academic middle; (2) evidence of 
a recruitment plan must be provided; (3) the school site team must be involved in recruitment; and (4) all 
students in the AVID elective must sign a contract and parents must participate in at least one meeting 
with school staff members focused on gaining parent support. 
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the level of attendance at each of the meetings by the site team members from the middle schools 
and high schools, and the development of a written plan for the feeder teams to bring back to their 
schools and follow. The four teacher content collaboratives were each scored on three elements 
each year: the facilitation of two meetings, the participation rate of teachers from both the middle 
schools and high schools at each session, and the development of a written plan. The cross-district 
vertical articulation collaborative (VAC) was scored on the facilitation of three meetings per year, 
the attendance at those meetings, and a written plan coming out of those meetings. The state college 
partnership’s implementation scores each year were based on the percentage of participation in the 
trainings (Summer Institute and local trainings) and the teacher content collaboratives compared 
with participation goals. For each alignment activity, scores of low/no, partial, or full implementa- 
tion were awarded for each measure, and total scores were tallied for each year. 


Implementation fidelity data were collected from the AVID systems coach twice per year. Data 
sources include the AVID Center’s Certification Self-Study documents for each school, which mea- 
sure the level at which each school is implementing the AVID elective and site team portions of the 
program, agendas and attendance rosters from various professional development events and team 
meetings, and other project documents. See Table 2.1 for a summary of all data sources and the 
timing of data collection. 


TABLE 2.1 
Data Sources and Timing of Data Collection, All Data Types 


Data Type 


Source 


Collection Timing 


Implementation fidelity 


Qualitative research 


School outcomes 


Student outcomes 


AVID Certification Self-Study 
(CSS) for each school, agendas 
and attendance rosters for 
professional development and 
meetings, and other project 
documents 


Interview and focus groups of 
teachers, other school and district 
staff members, middle and high 
school students, and college 
faculty members 


Interview with the AVID systems 
coach 


Survey of teachers, principals, 
and guidance counselors at all 8 
middle schools and high schools 


Survey of 10th-grade students in 
all 4 high schools 


Student-level school district 
administrative records for 
students in grades 8-12 


Winter and summer 2014, 2015, 
2016 


Summer 2013, fall 2014, winter 
2016 


Twice per school year, 2013-— 
2014, 2014-2015, 2015-2016 


Spring 2013 (pre-AVID year), 
2014, 2015, and 2016 


Early fall 2013 (start of Year 1) 
and 2016 (after Year 3) 


2010-2011 through 2015-2016 (3 
years before and 3 years during 
implementation) 
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Qualitative Research 


Qualitative data were collected and synthesized to learn about the implementation of the ACRS and 
alignment activities, to flesh out participants’ perspectives on the program and process of imple- 
mentation, and to learn about the obstacles to and enablers of implementation. For the qualitative 
research, interview and focus group data were collected from teachers, other school and district 
staff, middle school and high school students, and college faculty during three site visits: (1) during 
the first Summer Institute training, (2) at the beginning of the second year of the program, and (3) 
in the winter of the third year of the program. Qualitative research also included interviews with 
the AVID systems coach twice per year during the three years of the study. Interviewers asked 
participants about their experience with the program, the factors that enabled implementation, the 
obstacles to full implementation, and perceived changes in the school since the project’s inception. 


During the first site visit, teachers from all schools, principals from some schools, and some in- 
structors from the college participated in focus groups. For the second and third visits, two middle 
school-high school pairs were chosen to participate in the qualitative research. These middle schools 
and high schools were chosen because they represented the variety of the schools in the program. 
Two of the schools were in one of the small school districts and two were in the larger district. They 
also represented different levels of implementation strength during the first year and different types 
of challenges as identified by the AVID systems coach and by the first-year fidelity of implementa- 
tion data. Teachers, other school staff, and students participated in interviews and focus groups 
during visits to each of the schools. The school staff members who were interviewed included the 
principal or assistant principal, the AVID coordinator, the AVID elective teacher, and a focus group 
of teachers. A group of students also participated in a focus group at each of the four schools. The 
college AVID liaison and college teachers also participated in interviews or focus groups during the 
second and third visits. 


SCHOOL AND STUDENT OUTCOMES 


The school and student outcomes delineated in Chapter 1 were measured using student and staff 
surveys and student-level administrative records data collected at all the participating schools. In 
both cases, the study focuses on the outcomes of all teachers and staff members and all students 
across the schools. This group includes the AVID elective teacher and students, but they are just a 
small subsection of the entire school. This study focuses on the schoolwide effects of the program, 
not those particular to the AVID elective component. For school and student outcomes, the analyses 
compare the outcomes during the three years of implementation with the same measures in the years 
before AVID implementation at the same schools. 


These analyses are nonexperimental. That is, the study is not a randomized controlled trial in which 
the outcomes for two similar groups (one that receives the intervention and one that does not) are 
compared to determine the program’s effect. Additionally, this means that the study does not capture 
whether implementation of the program caused any change in outcomes. The school and student 
outcomes analyses simply compare the outcomes in the years before AVID implementation with 
the outcomes during the years of the program implementation to identify correlations only, without 
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looking at causation. Other changes in the schools’ policies, curricula, or student body could have 
also led to any changes in outcomes that are found. The implementation of the program was a siz- 
able undertaking and was probably the largest programmatic change at each of these schools during 
this time period, so any positive outcomes may point to the promise of the program to affect school 
and student outcomes. A more rigorous study, however, would be needed to measure whether the 
program has a direct impact on school or student outcomes. 


School Outcomes 


As discussed in Chapter 1, school outcomes include increased knowledge and use of the WICOR 
model (writing, inquiry, collaboration, organization, and reading) by all teachers, incorporation 
of AVID core values by school staff, and increased collaboration by school staff within and across 
schools. To measure increases in these outcomes, a staff survey was administered to all teachers, 
guidance counselors, and principals in all participating schools in the spring of 2013 (that is, the 
spring before AVID began implementation at the schools) and each spring during the three years 
of program implementation (spring of 2014, 2015, and 2016). The study team created the survey 
items in partnership with the AVID Center and designed them to capture teachers’ reported level 
of use of the WICOR model in their classrooms, the attitudes of school staff toward the AVID core 
values of providing academic rigor for all students and preparing all students for college, and the 
level of collaboration within and across schools. The measures that were included in the analysis 
were constructed by combining survey items. All the items in each construct were tested using fac- 
tor analyses to ensure they would hold together as a single measure when combined. (The items 
that make up each construct along with the Cronbach’s alphas for each construct are described in 
Appendix B.) The survey analysis compares the school averages for each measure from each year 
of program implementation with the findings from the survey that was administered in the spring 
before AVID was implemented. 


Survey items were compared over time to measure any growth in these outcomes and to answer the 
following research questions: 


e Over the three years of the ACRS and alignment activities implementation, do teachers across the 
participating schools report annual increases in knowledge about and use of the WICOR model 
in their classrooms? 


e Over the three years of the ACRS and alignment activities implementation, do teachers, guidance 
counselors, and principals at participating schools report annual increases in knowledge about 
and incorporation of AVID core values throughout their schools? 


e Over the three years of the ACRS and alignment activities implementation, do teachers, guidance 
counselors, and principals at participating schools report annual increases in collaboration to 


strengthen rigor and alignment of courses within and across the schools and the college? 


These school outcomes are considered mediating outcomes and are hypothesized to lead to the 
student outcomes discussed below. 
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Student Outcomes 


As discussed in Chapter 1, the hypothesized student outcomes for the program include increased use 
of learning skills, including study and reading strategies; increased engagement in school; increased 
expectations and awareness of postsecondary options; increased enrollment and credits earned in 
advanced courses; improved academic performance, educational attainment, and persistence in 
school; and strengthened college enrollment and persistence. The first three outcomes were measured 
using a student survey while the other outcomes were measured using student-level administrative 
records data. Given the confines of the study timeline, only college enrollment was measured.” The 
study was not able to look at persistence in college. Since the focus of this project was on systems 
change across the schools and districts, these analyses focus on the outcomes of all students across 
the schools as opposed to looking at the outcomes of only the AVID elective students. 


Outcomes from the Student Survey 


A survey was administered to all tenth-grade students at the four high schools in the early fall of the 
first implementation year of AVID (fall 2013), before they had much exposure to the program. The 
same survey was administered again to a new set of tenth-grade students during the fall after three 
years of implementation (fall 2016). Most of this second set of tenth-grade students experienced the 
program implementation during their seventh-, eighth-, and ninth-grade years in both middle school 
and high school. The survey asks students about their learning skills and strategies, including the 
reading and study strategies they use, their use of critical thinking in school, their study habits, and 
their use of organizational techniques. The survey also asks about their engagement in school, their 
awareness of postsecondary opportunities and expectations for their future higher education, and 
their perspectives on the school environment and the academic rigor of their classes. 


Tenth-grade students were chosen to participate in the survey because after three years of imple- 
mentation most tenth-grade students would have experienced the program for three years in both 
middle school and high school. They were also chosen because students in later grades would be 
more likely to drop out or leave the school, which could skew the sample. Two different sets of tenth- 
grade students were surveyed rather than surveying the same group of students in an earlier and 
later grade because as students progress into successively higher grades, their academic skills, college 
knowledge, and school engagement can change, regardless of any school program. Thus, it would 
have been difficult to interpret these survey findings from the same students in different grades. 


The measures included in the analysis are constructed from survey items. (See Appendix B for more 
detail on the survey analysis and the method for constructing the survey measures.) The analysis for 
the student survey compares the school averages in the pre-program tenth grade sample — that is, 
students who entered tenth grade in 2013 — with the school averages from the sample of students 
who entered tenth grade in the fall of 2016, after the third year of implementation for each measure 
had ended. 


2. College enrollment data used for this study only include counts of high school graduates who enrolled in 
college and not individual information about each student’s enrollment in college. 
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Outcomes from Administrative Records 


Measures of students’ advanced course taking, academic achievement, and educational attainment 
during middle school and high school were created using student-level administrative records data 
collected from the three school districts. These analyses address the following primary research 
questions: 


e Does the program offer promise in terms of strengthening the following outcomes after three 
years of implementation? That is, is it associated with these outcomes? 


° The likelihood that high school students across the school will enroll and succeed in more 
advanced courses (honors, Advanced Placement, and dual enrollment) during the ninth 
through twelfth grades 


° The average academic performance (grade point average and state assessments) of all ninth- 
through twelfth-grade students at the study schools 


° The overall educational attainment (credits earned, promotion to the next grade, and high 
school graduation) of all ninth- through twelfth-grade students at the study schools 


© Students’ persistence in ninth through twelfth grades (retention in school) for all students 
attending the study schools 


The study team also looked at a similar set of outcomes for eighth-graders at the end of the third 
year of implementation. Most of these students participated in ACRS for two years (during seventh 
grade in 2014-2015 and during eighth grade in 2015-2016). Since the program was implemented in 
middle schools only in the seventh and eighth grades during the first year, the ACRS was not offered 
to these students during their sixth-grade year in 2013-2014. 


Each district provided data on eighth- through twelfth-grade students’ course taking, grades received, 
and credits earned, their standardized test scores in English/language arts and math (including 
math and English/language arts for eighth-graders, and English/language arts for ninth- and tenth- 
grade students), whether they were promoted to the next grade at the end of the school year, whether 
twelfth-grade students graduated from high school, and whether students stayed in school the entire 
school year. Data were also collected on students’ demographic information — including race and 
ethnicity, gender, whether they were English language learners or participating in special educa- 
tion, and whether they were receiving free or reduced-price lunch (a proxy for low income) — and 
their prior scores on standardized state tests in English and math. These data were collected for all 
students attending the schools in eighth through twelfth grade during the three school years before 
AVID implementation (2010-2011, 2011-2012, and 2012-2013) and the three years after the program 
implementation began (2013-2014, 2014-2015, and 2015-2016) — that is, the three study years. 


The study team conducted analyses comparing the years before and after program implementation 
began using a pre-post analysis method to measure changes in outcomes over time at the participat- 
ing schools.* All four high schools were included in the analysis (and all four middle schools were 


3. See Appendix C for more information on the analytic model. 
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included in the eighth-grade analysis). For the high school analysis, students in ninth through twelfth 
grades were included for all measures appropriate for all grade levels. (Some measures, such as high 
school graduation, are limited to specific grades in the analysis.) Students are compared within each 
school and grade level, and then the findings for each school and grade level are combined. 


The study team also collected data on postsecondary education enrollment for students who gradu- 
ated some time during the three-year study period (discussed in Chapter 5). These data were not 
collected by the school districts and are not available to the study team for the years before the ACRS 
and alignment activities were implemented. Postsecondary records are compared over the three years 
of implementation with the assumption that given the rollout of the ACRS to lower grade levels first, 
high school seniors in the first and second years of implementation had less exposure to the system 
than those in the third year of implementation. 


The study sample for the administrative records analyses includes all four high schools that were 
participating in the study and their four associated middle schools. All students who were enrolled 
in ninth through twelfth grades at the end of the third implementation year are included in the main 
analysis. The eighth-grade analysis includes all eighth-grade students who were enrolled in the four 
middle schools at the end of the third year of implementation. 
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Implementation 


he implementation study examines the fidelity of the implementation of the AVID College 

Readiness System (ACRS) and the alignment activities to the project design, documents 

the experiences of both student and staff participants, and describes the contextual factors 

affecting implementation. This research was done through the collection, analysis, and 
synthesis of project data as well as via interview and focus group data from site visits to select study 
schools and phone interviews with the AVID project systems coach. 


Key findings from the implementation study include: 


e Overall, analyses show that two of the three ACRS components — professional development and 
the AVID elective — were implemented successfully at most schools, with moderately high fidelity 
to the model. The third component — site teams — had a slow start at many of the schools, but 
was moderately successful by the third year. 


e Incontrast, there was mixed success implementing the alignment activities, collaborative struc- 
tures that were developed specifically to increase alignment across schools and districts. 


e The biggest obstacles to successful implementation for the Central Florida Collaborative partners 
included teacher and administrator turnover, which made it hard at some schools to build the pro- 
gram over the years; a lack of tutors for the AVID elective, which made it difficult for some schools 
to fully implement that key component; and struggles to fully define and implement the teacher 
content collaboratives, which kept teachers across the schools from collaborating in robust ways. 


e The ACRS and alignment activities benefited from the strong support of the AVID systems coach, 
who worked to guide the program activities within and across schools; the active engagement of 
school administrators and the robust community of practice created by the collaboration across 
district and school administrators and some teachers; and the state college’s embrace of the 
ACRS, which led to more college faculty participating in AVID professional development than 
originally anticipated. 


Building College Readiness Across Rural Communities | 23 


FIDELITY OF THE AVID COLLEGE READINESS SYSTEM 
TO THE MODEL 


Fidelity of the ACRS implementation to the model was measured at the school level, focusing on three 
key system components (professional development, the AVID elective, and the site team) and 10 indi- 
vidual elements within those components: three elements of the planned professional development, 
six elements of the AVID elective, and one element of the AVID site team. Each element was scored 
on a scale of 0 to 2, with 0 equaling low/no implementation, 1 equaling partial implementation, and 
2 equaling full implementation with high fidelity to the model. Scores were assigned to each school 
each year and were combined across the eight participating schools each year to determine whether 
the project met an implementation threshold set by the AVID Center. Table 3.1 displays the number 
of schools, out of the eight middle schools and high schools, that met the full implementation goal 
for each element in each year. The first page of Appendix Table A.1 presents the operational defini- 
tion and data source for each element of the three ACRS components. 


Professional Development 


To initiate and support the successful implementation of the ACRS, the AVID Center planned to 
provide professional development to teachers and school leaders in the Central Florida Collaborative. 
The professional development focused on the WICOR model (writing, inquiry, collaboration, orga- 
nization, and reading). The goal for the professional development sessions was for teachers to leave 
the trainings with new skills, strategies, and tools that would help them increase academic rigor for 
students. The planned professional development took three main forms: the AVID Summer Institute, 
local AVID Path to Schoolwide trainings, and specially designed coaching sessions for AVID elective 
teachers and coordinators (as described in Chapter 1). 


Professional development was the most successfully implemented component of the project. Professional 
development and training sessions were delivered as planned (in some cases more frequently than 
planned), and a majority of schools usually met the participation benchmarks set by the AVID Center. 
The Summer Institute was the most successful component of professional development and the Path 
to Schoolwide training was the least successful. 


Summer Institute 


AVID Summer Institute trainings were successfully implemented as planned. Taking place over 
three days, the Summer Institute brings together staff from AVID schools across a geographic re- 
gion of the United States. The training sessions at the Summer Institute are divided into strands for 
schools that are newly adopting AVID and for those with more experience. Teams of teachers and 
staff attend from each school, and participants spend part of their time in workshops and part of 
their time in team debriefing and planning sessions so that they return to their schools with concrete 
plans to implement what they have learned. Summer Institute workshops are engaging and high- 
energy. AVID consistently uses adult learning theory, which promotes hands-on experiences and 
problem solving, in creating its professional development. In addition, the AVID Center demands 
that its presenters, who all are experienced AVID teachers, illustrate the WICOR model strategies 
that teachers are being trained to use. 
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TABLE 3.1 
Number of Schools Meeting Full Implementation Standard on 
Key Components and Elements of the AVID College Readiness 
System (ACRS), by Year 


Year 1 Year 2 Year 3 
Key Component or Element (2013-2014) (2014-2015) (2015-2016) 
Professional development 
Summer Institute 8 8 8 
Path to Schoolwide 5 
Coaching for AVID elective 8 8 7 
teachers and AVID coordinators 
AVID elective 
Class offering 8 8 8 
Recruitment and enrollment 7 7 8 
Organizational skills 5 7 8 
Inquiry and collaboration 5 6 7 
Reading and writing 4 6 6 
Tutoring 4 5 4 
Site team 2 5 6 


SOURCE: AVID certification documents for school years 2013-2014, 2014-2015, and 2015-2016. 
NOTE: There are eight schools in total. 


For this project, AVID provided space for each middle school and high school to send a team of 15 
teachers and staff to the Summer Institute during summer 2013 — before the first year of imple- 
mentation — and a team of 8 teachers and staff to the Summer Institute each summer thereafter. 
Participation was high: In the first year, each school sent between 13 and 17 attendees. In subsequent 
years, every school sent at least 8 teachers and staff, with several schools paying the additional fees 
to send a total of 9, 10, or even 11 participants. When asked about their impressions of the Summer 
Institute, teachers and staff were almost unanimously positive. One teacher interviewed by the re- 
search team shared her perspective: 


There is a lot of interaction and [we are] practicing all of the things that she’s teaching us, and 
when she’s got stuff on the wall [she says,] “Go take pictures. You can have it with you so that you 
can look back on it later” It’s just very open and safe. It definitely feels like a safe classroom where 
you can make a mistake and it’s okay, nobody’s gonna laugh at you, which is good because some 


teachers are worried . . . so having that presenter that shows you to just do it is fantastic. 


In fact, when interviewing teachers at their schools in the second year of the project, the research 
team heard some teachers express frustration and consternation over not having yet attended the 
Summer Institute — because they had heard from their peers that it was such a motivating and 
meaningful professional learning opportunity. It was in part the teachers’ willingness to dedicate 
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three days of their summer vacation to receiving high-quality professional development that ensured 
that the project successfully met the implementation threshold for participation. As can be seen in 
Table 3.1, all eight schools met the implementation threshold for attending the Summer Institute 
in all three years. 


Path to Schoolwide Training 


Local AVID Path to Schoolwide training sessions focused on a range of topics, including general 
implementation challenges, leadership strategies, the AVID tutorial program, and the use of stra- 
tegic engagement strategies in core classes. AVID committed to providing two full days of Path to 
Schoolwide professional development each year with spots for 10 teachers and other staff members 
to participate from each school. In fact, AVID responded to requests from the schools by offering 
far more training than planned in the first year of the project — up to six days at some schools. 
While AVID expected that each school would have a participation rate of at least 85 percent for the 
two days of required training, five of the schools met this expectation in the first year and seven of 
the schools met the expectation in the second year, but only four of the schools met the expectation 
in the third year of the project, as shown in Table 3.1. Therefore, despite the initial enthusiasm, the 
implementation of Path to Schoolwide trainings was only moderately successful. 


AVID Elective Teacher and AVID Coordinator Coaching 


The third and final type of professional development support provided by the AVID Center was a 
series of coaching sessions for the AVID elective teachers and the AVID coordinators. AVID pro- 
vided these coaching sessions once per semester and tailored them to the needs of the AVID elective 
teachers and AVID coordinators. AVID Center staff expected that the AVID coordinator and at least 
one AVID elective teacher from the school would participate each time. (In some cases, the same 
individual was both the teacher and the coordinator. In others, one or multiple individuals taught 
sections of the AVID elective class, while another individual coordinated the recruitment, student 
scheduling, hiring, and training of tutors; fundraising; and other tasks associated with managing 
the AVID elective program at the school.) In the first and second years of the project, the targeted 
individuals participated at least 85 percent of the time at all eight schools; in the third year, seven of 
the eight schools met the attendance goals. Despite this slight decline in the third year, the imple- 
mentation of the AVID coaching sessions successfully met the implementation goals. 


AVID Elective 


The AVID elective class, although it serves only a small percentage of the students in the school, 
is the cornerstone of the ACRS. AVID students attend the AVID elective class five days per week 
to learn critical reading, writing, and thinking skills; participate in collaborative tutorials led by 
trained college students; and engage in career and college exploration. The project plan stipulated 
that each school begin offering the AVID elective to select grades in the first year of the program 
and continue expanding the number of sections until at least one section was offered at each grade 
level by the third year of the project (forming an unbroken continuum of AVID elective sections in 
the sixth through twelfth grades in each middle school-high school pair). In addition, the schools 
were required to ensure that the students who enrolled in the AVID elective were the ones defined 


26 | Building College Readiness Across Rural Communities 


as the target group by the AVID Center (that is, students who were previously not high academic 
achievers, but who demonstrated some academic capability and motivation to succeed). 


Program data collected annually show there was notable variation in implementation fidelity by 
program element. The first three elements of the AVID elective (class offering, recruitment and 
enrollment, and teaching of organizational skills) were, for the most part, successfully implemented 
with fidelity to the model across the study sample: At least seven of the eight schools successfully 
implemented all three elements in the second and third years of the study; five schools successfully 
implemented organizational skills in the first year only. The remaining program elements (teaching 
reading and writing skills, teaching inquiry and collaboration, and use of the tutorial structure) were 
more challenging to implement, but even these program elements were well implemented at half or 
more than half of the participating schools (as shown in Table 3.1). In addition, implementation of the 
AVID elective program elements (except for tutorials) improved during the three years of the project. 


Figure 3.1 shows the AVID elective implementation scores each year separately for each school. Each 
school could receive a total possible score of 12 points; in other words, each school could receive a 
score of 2 for each element that was fully implemented, a score of 1 for each element that was par- 
tially implemented, and a score of 0 for low or no implementation. Implementation fidelity varied 
somewhat by school type, with high schools found to have slightly higher fidelity of implementation 
than did middle schools. However, the dominant finding is that the majority of schools did success- 
fully achieve full implementation of the AVID elective within three years and that the remaining 
schools were moderately successful. 


Students who participated in AVID electives at the four schools that the research team visited had 
many positive things to say about their experience. College awareness was one theme that the students 
stressed. For example, one AVID student said, “AVID is a great way to learn about college; you take 
trips to schools where we might want to go.” Her classmate added, “We do a lot of preparation in AVID 
to help us figure out what we want to do, what would benefit us...what [our] skills and interests are.” 


Another theme that the students stressed was the value of the organizational tools taught in the 
AVID elective, especially the use of three-ring binders to keep their class notes and homework 
organized and calendars to keep track of assignments. One student shared that “the binder helps 
[me] stay organized, and the calendar helps me keep track of deadlines.” Another student agreed, 
explaining that because of the organizational skills he learned through AVID, he now “turns as- 
signments in before the deadline.” One student admitted that he originally did not want to enroll 
in the AVID elective class because the easily recognizable binder seemed like “too much” — that is, 
too much effort and maybe too obvious a sign of a student’s investment in school. But after a year of 
unsatisfactory grades, this student enrolled in the AVID elective class in tenth grade and credited 
the required organization of the binder system with helping him to get ahead. 


The AVID Center’s theory of change maintains that students in the elective class will not be the only 
ones to benefit, but rather that over time those students and teachers will help promote a positive 
school culture — one that evidences student engagement, high expectations, and a sense of collec- 
tive responsibility. In several instances, the research team heard about the positive influence of the 
AVID elective teacher and students at the school. For example, students and teachers at one school 
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FIGURE 3.1 
Fidelity Scores for the AVID Elective, by School and Year 
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SOURCE: AVID certification documents for school years 2013-2014, 2014-2015, and 2015-2016. 


NOTES: Total scores are the sum of the six element scores (class offering, recruitment and enrollment, teaching of organization 
skills, teaching of reading and writing skills, teaching of inquiry and critical thinking skills, and use of tutorials). For each element, a 
school received a score of 2 for full implementation, 1 for partial implementation, and 0 for no or low implementation. 


shared that the AVID elective students had set a very positive example with their community ser- 
vice projects; at another school, teachers remarked that AVID elective students were setting a new 
standard of organization and preparation in their classes. 


AVID Site Teams 


Each school participating in the project was required to form an AVID site team to support imple- 
mentation of the AVID elective and adoption of WICOR strategies schoolwide. Ideally, the AVID 
site teams comprise individuals in specific roles, including the designated AVID coordinator, the 
AVID elective teacher(s), the school guidance counselor(s), an administrator (that is, a principal 
or assistant principal), and a representative from each core academic content area. Site teams were 
asked to address alignment of the AVID elective curriculum from one grade to the next, continued 
improvement of challenging program elements like the tutorial program, and strategic planning and 
communication to ensure AVID elective students have access to and succeed in rigorous academic 
courses. Finally, the AVID site teams were expected to produce a detailed plan for ongoing program 
implementation and improvement. 
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Although every school established a site team to some degree, only two of the schools implemented 
this component successfully in the first year. Specifically, the research team found that not all the 
site teams had all of the members and some site teams did not meet at least monthly as required. In 
addition, although a few teams attempted to address issues of access and equity, other teams focused 
on more basic programmatic elements like field trip planning and fundraising, and were not able 
to tackle more systemic challenges. Some AVID coordinators and teachers said that they typically 
spent their meeting time talking about fundraising. For example, one elective teacher said, “I’m also 
concerned about money to operate the class; we have to pay for paper and cartridges.” A coordinator 
said, “Everyone feels overloaded... We need to fundraise for . . . activities, t-shirts, college visits...” 


The professional development offered to school leaders, AVID elective teachers, and AVID coordi- 
nators eventually began to influence the AVID site team members’ vision of what their teams could 
accomplish. By the third year of the project, site teams were being more successfully implemented 
across the eight schools, and were being fully implemented at six of the schools. This means that 
meetings were being held regularly, the appropriate individuals were attending, and the site teams 
were taking responsibility for supporting the ACRS at the school in substantive ways. 


Overall, data show that implementation of ACRS components was successful at most of the schools 
and moderately successful at the remainder. 


FIDELITY OF THE ALIGNMENT ACTIVITIES TO THE MODEL 


This section addresses fidelity of the implementation of the four alignment activities to the model: the 
middle school-high school feeder teams, the teacher content collaboratives, the vertical articulation 
collaborative (VAC), and the partnership with the state college. Table 3.2 displays the implementation 
scores (low, partial, or full implementation) of the four feeder teams for each alignment activity in 
each school year. Each of the four alignment activities was designed to increase alignment of cur- 
riculum, standards, and AVID implementation in sixth grade through college. They were designed 
by the AVID Center and the Central Florida partners specifically for this project and were under 
development and evolving continually during the three years of the study. For this reason, the levels 
expected to meet fidelity were adjusted during implementation. However, the findings for each year 
shown in Table 3.2 are all measured using the same final set of expectations that were determined 
by the AVID Center, which reflect their best judgment about what was needed for effective imple- 
mentation. 


Middle School-High School Feeder Teams 


The first of the four alignment activities that the AVID Center developed for this project, the middle 
school-high school feeder teams, was intended to increase communication between the two schools 
in each middle school-high school pair, specifically regarding implementation of the AVID elective 
and the WICOR model schoolwide. The AVID Center originally expected that the two site teams 
for each middle school-high school pair would meet jointly at least once per semester and develop 
actionable plans to carry over from meeting to meeting (that is, plans indicating clear goals and 
activities, timelines, and individuals who would be responsible for carrying out the plans). Later, it 
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TABLE 3.2 
Achievement of Implementation Standards on Key Components of the 
AVID Alignment Activities, by Year 


Year 1 Year 2 Year 3 

Key Component (2013-2014) (2014-2015) (2015-2016) 
Middle school — high school feeder teams 

Feeder Team A Low Low Partial 

Feeder Team B Low Low Partial 

Feeder Team C Low Low Low 

Feeder Team D Low Low Partial 

Teams achieving full implementation (N) 0 0 0 
Teacher content collaboratives 

English/language arts Full Partial Full 

Mathematics Partial Partial Full 

Science Partial Full Partial 

Social studies Partial Full Full 

Collaboratives achieving full implementation (N) 1 2 3 
Vertical articulation collaborative (VAC) Full Partial Partial 
State college partnership 

Participation in professional development Full Full Full 

Participation in content collaboratives Partial Partial Partial 


SOURCE: Attendance rosters and document review in school years 2013-2014, 2014-2015, and 2015-2016. 


NOTES: There were four feeder teams, one for each high school and middle school pair. Implementation scoring for 
each team was based on the rate of attendance at meetings and the development of a written plan for the teams to 
bring back to their schools and follow. 


There were four teacher collaboratives, one for each of the four core content areas. Implementation scoring was 
based on the facilitation of two meetings per year, the teachers’ rate of attendance at those meetings from both the 
middle schools and the high schools, and the development of a written plan. 


There was one cross-district vertical articulation collaborative, which brings together administrators from all 
schools and districts, the college, and the consortium. The VAC was scored on the facilitation of three meetings per 
year, the rate of attendance at those meetings, and the development of a written plan. 


There was one state college partnership; implementation scores were based on the rate of attendance 
from designated college staff members in both the professional development trainings and the teacher content 
collaboratives. 


For each alignment activity, scores of low, partial, or full implementation were awarded for each measure. 


was decided that to meet full implementation, just two members from each site team — the principal 
(or assistant principal) and the AVID coordinator — would make up a feeder team (see Figure 3.2) 
that would meet only once per year and would still work to create actionable plans to bring back to 
the full site teams and the rest of the staff members at the schools. This component was implemented 
with increasing, but still relatively low, fidelity across the three years. 
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FIGURE 3.2 
AVID Site Team and Feeder Team Composition 


MIDDLE SCHOOL SITE TEAM HIGH SCHOOL SITE TEAM 
Principal, AVID coordinator, Principal, AVID coordinator, 
AVID elective teacher, other AVID elective teacher, other 

teachers, guidance counselor teachers, guidance counselor 


FEEDER TEAM 


Principal and AVID coordinator from middle school site team 
Principal and AVID coordinator from high school site team 


NOTE: Either the principal or the assistant principal from each school could serve on the site teams and 
feeder teams. 


In fact, the feeder teams met sporadically, if at all, in the first and second years of the project. By the 
third year, three of the four feeder teams were holding regular meetings with focused agendas, but 
none of the teams was consistently developing and using actionable plans. As can be seen in Table 
3.2, none of the four feeder teams met full implementation in any year, and all four teams had low 
implementation in the first two years. Interviews with school and program staff indicate potential 
feeder team members were in many cases unclear on the purpose of the feeder team meetings. In 
addition, the functionality of a successful middle school-high school feeder team would clearly be 
dependent upon relatively strong AVID site teams at both schools. Since it took until the third year 
for the site teams to become truly functional at most of the schools, it is not surprising that the 
middle school-high school feeder teams were not successfully implemented. 


Teacher Content Collaboratives 


The second alignment activity that the AVID Center and its partners developed for this project was 
the teacher content collaborative. Content collaboratives were designed to bring together teachers 
from all eight schools within a core academic content area: one for math, one for science, one for 
English/language arts, and one for social studies. Participants met to discuss implementation of 
AVID instructional methodologies within the content area and worked to align content and strategies 
across grade levels. The AVID Center intended that teacher content collaborative meetings would 
take place once a semester for a full day and that a prescribed number of teachers per content area in 
a school would participate. To ensure that the collaborative conversations included all schools and 
districts, the fidelity of implementation to the model was measured in part by how many schools 
were represented at each collaborative meeting. Finally, each collaborative was expected to develop 
an actionable plan, meaning that specific goals and activities would be documented, with timelines 
and responsible individuals indicated for each. 
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As can be seen in Table 3.2, the partners were only partially successful in implementing the teacher 
content collaboratives in the first and second years of the project, but adjusted over time and were 
successful in meeting most implementation standards for the teacher content collaboratives in the 
final year. The AVID systems coach took primary responsibility for facilitating the teacher content 
collaborative meetings, so the meetings were held according to schedule all three years, but at- 
tendance and participation were inconsistent. For example, the AVID Center originally expected 
schools to send three representatives per content area to each meeting, and several schools were 
unable to do so because of a shortage of available substitute teachers and its impact on instruction. 
As one principal explained, “[F]inding subs in rural areas is hard and the standard is low because of 
the shallow pool. We had to cancel [a teacher content collaborative] because we couldn’t find subs.” 
The AVID systems coach noted these challenges and made several revisions to the plan, including 
reducing the number of teacher participants from each content area at each school from three to 
two, as well as attempting to construct a schedule for the content collaborative meetings that had 
less impact on the instructional day. As can be seen in Table 3.2 (which uses the revised criteria of 
two teacher participants per school), while only one collaborative (English/language arts) was able to 
meet full implementation in the first year, by the third year three collaboratives (English/language 
arts, mathematics, and social studies) met full implementation. 


Vertical Articulation Collaborative 


The vertical articulation collaborative was the third alignment activity that the AVID Center devel- 
oped for this project. The VAC met three times per year to address the alignment of curricula and 
teaching strategies across grade levels and across the participating schools and districts and to support 
implementation of AVID activities and core values from sixth grade through college. Designed to 
support collaboration at the leadership level, the group comprised administrators (school principals 
or assistant principals), the AVID directors from each school district office, representatives from 
the state college, and a representative from the regional consortium supporting the grant. At the 
outset, as many as 16 different individuals were expected to participate in the VAC. Eventually, the 
AVID Center staff streamlined the list of intended participants down to 13 individuals. Using the 
revised criteria, the VAC successfully met the attendance thresholds in the first year, but was only 
partially successful in meeting attendance goals in the second and third years, as shown in Table 3.2. 
The AVID systems coach noted, however, that while attendance of school administrators declined 
in the second and third years, participation by teachers increased. Boosting teacher leadership is an 
explicit goal of the ACRS, so their attendance is viewed as beneficial. 


In addition, the AVID systems coach and school administrators who were interviewed by the research 
team believed that the VAC became a valuable community of practice for school leaders. The practice 
of conducting classroom walk-throughs at one another’s schools was cited for building engagement 
and camaraderie among the principals. AVID Center staff had worked with members of the VAC 
to design a classroom walk-through protocol early in the project; the protocol was an observational 
tool that was intended to guide members of the VAC as they toured their host school and visited 
classrooms; the protocol also included a series of questions and checklists to prompt observation 
and discussion about the characteristics of the teaching and learning they were observing. The 
classroom walk-throughs were cited as a powerful learning experience by school and district leaders 
and participating teachers. One principal stated: 
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It’s powerful when teachers see themselves in a situation in a classroom thinking, “I do that, too,’ 
but as an observer they see that it's not that effective, or they see “what they are doing is more 
effective than what I do.” One [teacher] just realized something that she’s been doing in her class- 
room ... how she’s doing the work and not her students. . . “lifting the weights” as we say. It was a 
world of difference on the walk-through. She gets it. I was anxious to get back into the classroom 
to see how it translated, and [we] walked out saying, “The students are doing the work in that 


classroom now. 


The principal further explained that the walk-throughs provided teachers with the opportunity to 
reflect on their practice in a safe environment. He said that the debrief following the walk-through 
spurs productive conversation and improves the instructional environment. 


To build on these successes, the AVID systems coach decided to continue using walk-throughs as a 
tool for observation, reflection, and continuous improvement throughout the project. VAC meetings, 
therefore, had a routine structure that provided clear roles and responsibilities for participants, and 
plenty of data on which to base decisions and plans for future action. Although these plans were 
not always written in a form that AVID would consider actionable (for example, with goals, tasks, 
timelines, and responsible individuals), it was clear that the VAC meetings offered the opportunity 
for school leaders to hold meaningful conversations and accomplish meaningful work. 


State College Partnership 


The goal of the ACRS is to increase enrollment and success in college among the targeted student 
population. To facilitate better alignment of academic expectations and smoother transitions for 
students between high school and college, the AVID Center involved the local state college as a 
key partner in this initiative. The state college also hosts some of the dual enrollment courses, in 
which students can earn high school and college credit at the same time. Having the dual enroll- 
ment instructors at the college trained in the WICOR model could help high school students with 
their transition into these courses because the students are also using WICOR model strategies in 
high school. The state college was expected to extend the AVID project to its campuses by having 
its leaders and faculty participate in professional development and training in the teacher content 
collaboratives and in the VAC. 


Meeting agendas and attendance records indicate that the college’s involvement varied by activity: 
Participation in training and professional development was greater than expected, participation 
in the teacher content collaboratives was lower than expected, and participation in the VAC was 
inconsistent. For example, the stated goal of the project was to provide at least three days of profes- 
sional development to 60 college staff members in total over the three-year study period, or about 20 
staff members per year. In the first year of the project, over 20 staff members attended the three-day 
Summer Institute. Many of these staff members, along with another 10 staff members, also attended 
other training events offered during the first school year. In the second year, college leadership 
decided to pursue recognition as an “AVID for Higher Education” institution — a special certifica- 
tion that can be earned from the AVID Center. Faculty on the college’s AVID site team shared their 
own AVID implementation experiences with colleagues through a newsletter and open classroom 
demonstrations. One faculty member documented that even some of the instructors who had been 
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traditionally resistant to change became interested in the WICOR model: “When we talk about 
the engagement component to retain information, that’s more meaningful to them. . . Curiosity is 
being piqued. They want in to learn more.” The college subsequently provided six to eight days of 
AVID professional development per year on site, often attended by 20 college instructors or more. 
Therefore, the AVID core values and WICOR model were being suffused throughout the college. In 
addition, the college provided initial training in AVID instructional strategies to its own student 
education majors so that prospective teachers in the region would be familiar with them. Therefore, 
the college’s implementation of AVID professional development was highly successful and went 
beyond the original expectations. 


However, many of the teacher content collaboratives were short the two college staff members who 
were supposed to attend, and college staff were also absent from some of the VAC meetings, making 
the college’s participation in the alignment activities only partially successful. Interviews conducted 
with the AVID systems coach, the college dean, and college instructors indicate that it was challeng- 
ing for college instructors to make time for such meetings in their schedules. The AVID systems 
coach explained that these scheduling challenges were unanticipated: 


The real hurdle is the time of [the teacher content collaborative]. It doesn’t always fit for the 
college; they would have to cancel their class. It’s not a good model for faculty to be involved ... 
it's a struggle to come for the whole day when they have three to four classes in the day. It’s not 


something that was thought of, how faculty had challenges getting there. 


The college partnership was not implemented as originally intended because the college staff members 
did not fully participate in the alignment activities, which were meant to foster collaboration between 
the college and the secondary schools and school districts. Still, interest in the WICOR model and 
AVID’s core values was substantial across the college, as reflected by the strong participation in the 
AVID professional development. Since the college faculty, like their high school counterparts, were 
learning and likely using the WICOR model in their classes, including the dual enrollment courses 
taken by students from participating high schools, these students were more likely to have continuity 
in the teaching and learning strategies used across their classes at both the high school and college. 


FACTORS THAT ENABLED AND IMPEDED PROJECT 
IMPLEMENTATION 


Site visits to the AVID Summer Institute as well as two project high schools and their feeder middle 
schools conducted in the second and third years of the program, regular interviews with the AVID 
systems coach, and project data collected annually provided the research team with multiple data 
points and perspectives on the course of program implementation across the three years of the 
project. The team identified a few substantial obstacles to implementation and several important 
factors that may have enabled success. 
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Obstacles to Implementation 


Although the AVID Center provided substantial professional development and training during the 
years of the study, a notable challenge to implementing the ACRS to the desired level of fidelity was 
turnover and mobility rates among teachers and principals. Teachers and principals often switched 
(or were transferred by supervisors) from one role to another or from one school to another, which 
in some cases made it challenging to select and retain good AVID elective teachers, or to develop 
and sustain a strong site team.’ At some other participating schools, school leadership changed 
unexpectedly during the course of the project, resulting in lost momentum. For example, a teacher 
interviewed at the Summer Institute shared, “The original principal that brought [AVID] to us is 
not the principal we ended the year with...so that whole leadership process has just been very cloudy 
and blurred.” It was similarly challenging to maintain staff in the AVID elective teacher and AVID 
coordinator roles at some of the schools, which meant that affected schools had some difficulty 
maintaining consistency and momentum in implementing the program. In one case, a school had 
to recruit new students for the AVID elective class because so many dropped out when assigned to 
an AVID elective teacher who was not a good fit for the role. 


Another notable challenge in this project was implementation of the AVID tutorial. The tutorial 
is a key element of the AVID elective; it has a complex design intended to foster specific academic 
study skills and self-advocacy skills in students. The design relies on the availability of college or 
college-educated adult tutors who work part time for the school district. With considerable planning, 
coordination, and support, the tutoring component of the AVID elective program was implemented 
at most study schools by the end of the second year, but some schools struggled with the ongoing 
effort of recruiting and retaining tutors from semester to semester, and in some cases high school 
juniors and seniors acted as tutors when college students were not available. 


A third significant challenge of implementing the project as planned was the design of the teacher 
content collaboratives. The content collaboratives suffered from a lack of definition and focus as well 
as attendance challenges. Schools were unclear about whether they should send the same teachers 
to the collaborative meetings each time or rotate them so that different teachers could participate. 
With different members at each meeting, the content collaboratives were unable to progress in 
discussion and decision-making. As more than one teacher commented, “[We spend] a lot of time 
on team building because it’s not a consistent group.” In addition, the design stipulated that at least 
two teachers attend the full-day meeting from each school, which required that eight core content 
area teachers be out of school for a full day during the same week. Some schools struggled to ad- 
dress the staffing gaps created by the teachers’ absence or to grapple with the overall effect on the 
school of having teachers missing class time. Several principals reported that although the districts 
had grant money set aside for substitute teachers, there were not enough qualified substitutes to go 
around. Because the content collaboratives met each semester, these logistical challenges had to be 
confronted twice per school year. And some teachers reported that they felt conflicted about com- 
promising instruction for two full days per year. Finally, some teacher participants were confused 


1. Since teacher retention was not a specific goal of the project, quantitative data on teacher turnover and 
mobility were not collected. However, the issues of teacher turnover and mobility were raised on site visits, 
where researchers observed first-hand that many roles were filled by different individuals on the second 
site visit. 
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about what the content collaboratives were meant to achieve. To address these concerns, the AVID 
Center experimented with holding collaborative meetings outside of school hours, provided more 
direction regarding attendance and participation in the content collaboratives, and encouraged 
principals to follow up on the plans developed by the teachers at the content collaborative meetings. 


Enablers of Implementation 


Numerous factors seemed to support the more successful aspects of implementation in the AVID 
Central Florida Collaborative. These factors may be responsible for some of the project’s positive 
outcomes, and therefore they may require attention in the context of any replication efforts. The 
supportive factors included the resources and support of the AVID systems coach, the influence of 
strong school leaders, the successful evolution of the VAC into a genuine community of practice, 
and the state college’s substantive adoption of AVID. 


Two different individuals served as the AVID systems coach over the course of the three years of 
implementation; each was an extremely experienced and skilled AVID professional developer. The 
systems coach was assigned nearly full time to the three districts in the study, which was a much 
more significant amount of support than a typical AVID district would receive. In addition to all the 
planned training, professional development, coaching, and meeting facilitation, the systems coach 
was able to deliver additional professional development sessions and conduct additional classroom 
walk-throughs as needed. Such proximity allowed for development of integral relationships at all 
levels. In several cases, the AVID systems coach was able to coordinate with school or district leaders 
to support identification and placement of the most qualified individuals to serve in AVID-related 
positions at the schools. The expertise of the AVID systems coach, leveraged intensively for the 
duration of the project, was fundamental to successful project implementation. 


Although this finding likely comes as no surprise, the different school administrators were variously 
engaged with AVID and exhibited varying degrees of ability to lead their staffin implementing AVID. 
For example, some principals became well versed in the WICOR model strategies, and they made 
sure that their staff meetings and professional development days promoted implementation of the 
ACRS; others who were interviewed by the research team were much less fluent in the language of 
AVID. The AVID systems coach routinely described the varying levels of engagement and enthusiasm 
by principals across the three districts and noted that staff buy-in tended to vary according to the 
level of commitment to the project that the principal demonstrated. 


The VAC, which was the alignment activity most frequently cited as valuable by its participants, also 
played a significant role in implementation of the ACRS and other alignment activities. According 
to the principals, who were its primary participants, the VAC became a valuable forum for explor- 
ing the nuances of the program. By conducting walk-throughs to observe a variety of classrooms 
in person, principals were able to develop a common language for and an understanding of how 
various AVID strategies looked when implemented well, the common pitfalls, and how to problem- 
solve with teachers to improve their use of various strategies. In many education reform programs, 
school leaders never develop such a deep understanding of the program and are therefore unable to 
truly support teachers in implementing it. The VAC was also a place where principals talked about 
other aspects of leadership, including staffing, school climate, and so forth. In some districts, school 
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leaders have no forum for such discussions; in others the forum exists but it is facilitated and closely 
moderated by the principals’ supervisors. With the VAC, AVID’s goal was to develop a collaborative 
structure that would ultimately be “owned” and “driven” by the principals themselves, which likely 
supported their ownership of the initiative at their schools as well. 


A final factor supporting implementation of the ACRS across this rural central Florida area was 
the partnership with the state college. As noted above, the college’s participation in AVID profes- 
sional development and training was more substantive than had originally been proposed. Because 
the college dean became very invested in AVID, the AVID systems coach was able to provide more 
training and professional development at the college than originally planned. Then the college began 
pursuing status as an AVID Higher Education Institution, which involved the college developing its 
own site team and undergoing its own certification process through the AVID Center. Taking that 
step meant that the college was attempting not just to build a stronger bridge from high school to 
college, but to change its own internal culture to better foster student achievement. 


The AVID Central Florida Collaborative was designed to produce both school-level and individual- 


level outcomes. The next chapter of this report discusses the school-level outcomes during the three 
years of project implementation. 
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School Outcomes: Adoption of AVID Teaching 
Methodologies, AVID Core Values, and 
Collaboration by School Staff 


he school outcomes that are measured in the AVID evaluation are mediating outcomes 

hypothesized to come directly from the implementation of the AVID College Readiness 

System (ACRS) and alignment activities, and that lead to the student outcomes. The school 

outcomes include increased knowledge and use of the WICOR (writing, inquiry, collabora- 
tion, organization, and reading) model, by all teachers; increased incorporation of the AVID core 
values of academic rigor and college readiness for all students by teachers, guidance counselors, 
and administrators across each school; and increased collaboration among staff members within 
and across schools. 


Key findings from this chapter include: 


e There was a positive change in teachers’ reported use of most of the WICOR model teaching 
methodologies during the first year of implementation. This change was maintained for the most 
part across the three years of implementation. 


e School staff members’ reported attitudes and actions related to the AVID core values of academic 
rigor and college preparation were stronger during the three years of implementation compared 
with the year before implementation. 


e Reported levels of collaboration within schools and across the schools, districts, and college also 
grew during the years of the implementation. 


Table 4.1 displays the school outcomes based on a yearly survey of teachers, guidance counselors, and 
principals.! The survey was administered during the spring before the first year of implementation 
and during the spring of each of the three study years (2014, 2015, and 2016).* Each measure in the 
table is the mean for all the teacher responses within each of the eight schools, averaged across all the 


1. “Principals” includes both head principals and assistant principals, who are also referred to as school 
administrators in this report. 


2. Response rates for the survey ranged from 80 percent to 83 percent across the years. See Appendix Table 
B.1 for more information on response rates. 
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TABLE 4.1 
School Outcomes: Composite Staff Survey Measures, 


All Schools 
Mean Score 
Pre-AVID Year 1 Year 2 Year 3 

Composite Measure Year (2013-2014) (2014-2015) (2015-2016) 
WICOR model? (1 = never, 7 = daily)° 

Writing 3.14 3:63*** 3.76°** ou"? 

Inquiry 4.61 4.60 4.68 4.59 

Collaboration 3.18 350%" 3.49*** 3.48*** 

Organization 3.35 3.74*** S75." 3:83°** 

Reading 3.69 3.91" 3.86 3.82 
AVID core values (1 = strongly disagree, 
4 = strongly agree) 

Academic rigor 2.96 3.05** Bal2*** 3.11*** 

College preparation® 2:19 2:02, 2.41*** 2.42*** 
Collaboration (1 = never, 6 = daily)? 

Meetings within and between schools 1:91 2416 220°" 233°°° 


SOURCE: Surveys of teachers, administrators, and guidance counselors administered in May 2013, May 2014, May 2015, 
and May 2016. 


NOTES: Across the eight study schools, there were 427 participants in the spring before implementation, 434 participants in 
Year 1, 435 participants in Year 2, and 421 participants in Year 3. 


A two-tailed t-test was used for all statistical tests. Asterisks after the means for “Year 1,” “Year 2,” and “Year 3” represent the 
Statistical significance of the difference between that year and the pre-AVID year. Statistical significance levels are indicated as 
follows: *** = 1 percent; ** = 5 percent; * = 10 percent. 


Data are missing for no more than 10 percent of respondents on any measure. 
‘The WICOR model includes teaching strategies in writing, inquiry, collaboration, organization, and reading. 
°A score of 3 is equal to once a month and a score of 4 is equal to more than once a month but not weekly. 


‘This measure includes only one survey item: Staff members reported that all students at their school can be successful in 
college. 


4A score of 2 is equal to at least once a month. 


schools. The first column in the table shows the average score from the survey that was administered 
in spring 2013 (“Pre-AVID Year”), before the ACRS and alignment activities began at the schools, 
and the next three columns show the average scores from the survey that was administered in the 
spring of each year of implementation. The asterisks indicate whether the change from the mean of 
the pre-implementation year to the mean for each implementation year is statistically significant 
(meaning that the probability is low that there is no true difference between the pre-implementation 
year mean and the implementation year mean — that is, that any differences are not simply chance 
occurrences). See Appendix B for the Year 3 findings for each school. 
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USE OF THE WICOR MODEL 


The first section of Table 4.1 displays the scores for each of the five domains of the WICOR model, 
which are described in Chapter 1 (Box 1.2). The WICOR model represents the AVID teaching meth- 
odologies, which include a set of tools that teachers can use to create a rigorous college preparatory 
environment in their classrooms. For each domain, teachers were asked how often they used four 
to nine different specific strategies with their students. All of the strategies included are taught in 
AVID’s professional development component. For instance, teachers were asked about five different 
writing strategies and eight different organizational strategies.° Since the measures of strategy usage 
show the mean usage across a set of specific strategies rather than the total usage of all the strategies, 
the expectation would not be for teachers to be using each strategy every day. Expectations for the 
mean usage of these strategies are between once a week and once a month. 


On average, teachers’ reported use of writing, collaboration, organization, and reading strategies 
grew by the end of the first implementation year. During this first year many teachers received ini- 
tial training in these strategies. The level of reported use of these strategies generally held steady in 
the second and third years of implementation, with reading being the one exception and dropping 
slightly in the second and third years. As noted in Chapter 3, during interviews and focus group 
sessions, some school staff members reported heavy teacher turnover during the three years. Even 
with these staffing issues, schools were able to maintain the use of these strategies. (See Appendix 
Table B.3 for the third-year WICOR model outcomes by school.) On average, teachers reported using 
each of the specific strategies in these domains more than once a month. 


On average, teachers’ reported use of inquiry strategies did not change much during the three years 
of implementation, but teachers had reported relatively high use of these strategies even before 
implementation began. On average, teachers reported using the inquiry strategies slightly less often 
than weekly. 


AVID CORE VALUES 


The second section of Table 4.1 reflects teachers’, guidance counselors’, and principals’ reported 
attitudes and actions related to the AVID core values of academic rigor and college preparation for 
all students during the year before and three years of implementation. One of the key goals of the 
ACRS is to create a school environment in which all students, including low-income and minority 
students, are expected to do well in their classes and are given the tools and skills to succeed in 
rigorous course work. 


Academic Rigor 


The measure of academic rigor asks staff to indicate their level of agreement with each of five state- 
ments: (1) the school recognizes and acknowledges the academic achievement of its students; (2) the 


3. Each outcome measure is created by taking the average of all the strategies under that domain. See 
Appendix B for more information about the specific survey items that make up each outcome measure. 
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school values students’ academic achievement; (3) staff members feel students at the school try to 
get good grades; (4) staff members feel students at the school respect other students who work hard; 
and (5) staff members have a shared understanding of what students should know when they enter 
and leave each grade. Staff members were asked to rate these five statements on a four-point scale 
of “strongly disagree” (1) to “strongly agree” (4). On average, school staff members’ reported slightly 
higher levels of academic rigor at their schools during the years of implementation in comparison 
with the year before implementation began. (See Appendix Table B.4 for the findings in the third year, 
by school.) On average, staff members agreed that their schools recognized academic achievement, 
that students worked hard, and that there was shared understanding about the level of academic 
attainment students should meet in each grade. 


College Preparation 


In the ACRS, a key factor in creating a rigorous and supportive academic environment for all stu- 
dents is in creating a school culture that promotes the idea that students from all backgrounds can 
succeed in college. The measure of college preparation represents the attitudes of staff members in 
regard to students’ college readiness, and includes a single survey item that asks teachers, guidance 
counselors, and principals to indicate how strongly they believe that all the students at the school 
can be successful in college, on a scale of “strongly disagree” (1) to “strongly agree” (4). Staff mem- 
bers’ attitudes about the possibility of college success for all students grew during the course of 
implementation, but the average score for staff members during the implementation years still fell 
between disagreement and agreement. 


COLLABORATION 


The final section of Table 4.1 displays a measure of school staff members’ collaboration with other 
staff members at their own schools and at other schools, including the college. The measure is the 
average score based on teachers’, guidance counselors’, and principals’ responses to questions about 
their participation in meetings with other colleagues in their own school and at other schools within 
the district and across districts and the college. The meetings were convened to discuss students’ 
needs and access to rigorous course work, and to align methodologies, expectations, and curricula 
across grade levels. On average, staff members’ reported that collaboration increased during the 
three study years. Staff members reported having each type of collaborative meeting with different 
colleagues an average of at least once per month, suggesting some consistency with collaboration 
but not a fully robust system of collaboration. 


CONCLUSION 


During the first year the ACRS and alignment activities were implemented at the participating 
schools, many of the school outcomes showed positive change on expected dimensions of teacher 
practice, expectations of students, and staff collaboration, and most of these changes were main- 
tained in the following two years of implementation. As noted in Chapter 3, during interviews and 
focus group sessions, some school staff members reported high levels of teacher and administrator 
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turnover during the three years. Even with this staff turnover, most schools were generally able to 
sustain the progress made in the first year for the following two years. This finding strongly sug- 
gests that the ACRS and alignment activities took hold in the schools during the implementation. 
Still, teachers were asked to share how often they used WICOR model strategies, and school staff 
members were asked only about general attitudes toward college preparation and classroom rigor 
and how often they collaborated. From these analyses, it is not possible to gauge the quality of the 
implementation and any changes in the quality of implementation over the years of the study. These 
school outcomes were hypothesized to lead to positive outcomes for student behaviors, attitudes, and 
academic performance. Chapter 5 presents a discussion of the student-level findings. 
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Student Outcomes: Behaviors, Attitudes, and 
Academics 


he implementation of the AVID College Readiness System (ACRS) and alignment activi- 

ties is hypothesized to lead to the mediating school outcomes (discussed in Chapter 4), 

which are, in turn, hypothesized to lead to positive changes in students’ learning skills 

and study habits, attitudes toward school, and academic achievement and persistence. This 
chapter compares the outcomes of all students who attended the participating AVID schools before 
implementation began with the outcomes of all students who attended the schools during the three 
years of implementation. Although exploring these outcomes may shed light on the promise of the 
ACRS and alignment activities to meet the hypothesized theory of change laid out in Chapter 1, this 
nonexperimental study cannot identify the cause of any changes found in student outcomes during 
this period. The implementation of the ACRS was likely one of the largest systematic changes at 
the schools during the three years of implementation, but other factors such as changes in school, 
district, or state policy; curricular changes; or changes in the student body could also have triggered 
growth or decline in student outcomes. 


Key findings from this chapter include: 


e Little difference was found in the reported study habits, engagement in school, and postsecondary 
expectations between all the tenth-grade students at the school who were surveyed at the start of 
implementation and those who were surveyed after three years of exposure to the system, but, on 
average, both groups had relatively positive responses on most of the measures. 


e On average, across all students in ninth through twelfth grades at the high schools, students dur- 
ing the third implementation year were more likely to take advanced courses, earn at least one 
credit in an advanced course, and earn more advanced course credits than were students during 
the years before implementation. 


e On average, across all eighth-grade students attending the middle schools, there was a positive 
change in the percentage of students taking an honors course and earning at least one honors credit, 
and in the average percentage of honors credits earned, between students attending eighth grade 
before implementation and those attending eighth grade during the third implementation year. 
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e There is little difference in the two groups of students (those in the year before implementation 
and those in the third year of implementation) in their academic performance, educational at- 
tainment, and high school persistence. 


e Although each high school shows some fluctuations over the three years, there is little difference 
across all four high schools in the level of students’ postsecondary enrollment in the fall after 
high school graduation for those students who graduated from high school at the end of the third 
year of implementation and who experienced three years of the system, compared with students 
who graduated from high school at the end of the first and second years of implementation and 
who were in grades that were not targeted for ACRS implementation. 


STUDENT OUTCOMES IN LEARNING SKILLS, ENGAGEMENT, 
AND COLLEGE ASPIRATIONS 


The student survey was administered to tenth-grade students across the four high schools at two 
different times. First, it was administered to tenth-grade students who were attending the four par- 
ticipating high schools in the early fall of the first implementation year (2013) before those students 
had much exposure to the ACRS. The same survey was administered again to a new set of tenth- 
grade students who were attending the four high schools during the fall of 2016, after the ACRS 
and alignment activities had been implemented for three years. Most of the students in this second 
group had experienced three years of implementation starting in middle school during seventh 
grade and running into their ninth-grade year at the high school. Participation in the survey was 
high during both administrations, and approximately 86 percent of tenth-grade students early in 
Year 1 and 89 percent of tenth-grade students after Year 3 participated in the survey. (See Appendix 
Table B.2 for more detail on response rates.) The survey measures students’ reported use of learning 
skills, engagement in school, expectations and awareness of postsecondary education options, and 
perceptions of the school environment. 


This survey has some limitations. First, it does not measure growth over time for the same group 
of students, but instead compares different groups of students. This was a purposeful choice by the 
study team. If the same students were surveyed at different times, it would have been difficult to 
interpret findings on postsecondary awareness and expectations, since students’ knowledge gener- 
ally increases naturally over time and they might come to clearer decisions about postsecondary 
education options as they proceed through high school, regardless of their exposure to the ACRS. 
Still, surveying different students at different times also has limitations. For one, the reference point 
for students who are surveyed at the end of the study may have changed in comparison with those 
who were surveyed at the beginning of implementation. Since the entire school was exposed to the 
ACRS, many students may have improved their use of study skills and their engagement in school, 
so that students taking the survey during the two different periods are comparing themselves with 
a different status quo. 


The time of the administration of the survey may also have had some effect on the outcomes. In 


order to capture students’ outcomes before much exposure to the system, the survey administration 
occurred in early fall 2013 and this timing was matched for the second survey administration (in 
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early fall 2016) to ensure similarity between the two survey administrations. Even if students had 
changed their study habits and engagement over the course of the three years of exposure to the 
system, it is possible that some of the effects of these changes would have worn off or been forgotten 
over the summer, making any outcomes reported at this period weaker than they may have been if 
the survey had been administered at a different time in the school year. It is also possible that in the 
fall of 2013, teachers trained in the WICOR model during the first Summer Institute (the three-day 
professional development event for AVID teachers) were already exposing their students to these 
strategies. Still, the students who participated in three years of implementation had a lot more ex- 
posure and time to learn and embrace the strategies than those surveyed in the fall of the first year. 


Use of Learning Skills 


The first section of Table 5.1 displays the reported use of learning skills that are part of the ACRS, by 
all tenth-grade students across the four high schools.' Teachers are trained in these learning strate- 
gies through professional development and then can use them in their classrooms and share them 
with students. For the first measure, “Frequency of use of reading and study strategies,” students 
were asked how often they used eight different strategies in their studies. For instance, they were 
asked how often they test themselves by summarizing in their own words something they had read 
and how often they create diagrams and pictures to help them remember information. Students were 
also asked how frequently they use critical thinking activities in their classes and homework, such 
as how often they apply what they learned previously to what they are currently learning. The third 
measure included items asking students about their frequency of use of study habits such as writing 
summary sentences of passages they read or grouping their notes into three columns of questions, 
facts, and steps. 


For all these measures, there was little difference in the responses of students at the beginning of 
implementation and after three years of implementation. But in each case, students in both groups 
reported relatively high rates of using these learning skills, reporting, on average, that they use each 
strategy more than “sometimes” or more than once or twice a month. Ideally, some of these strate- 
gies could be used more frequently, but these scores suggest that students are using some kind of 
strategy quite often. 


There was also little difference in students’ reported frequency of use of three-ring binders and cal- 
endars, two key organizational strategies of the ACRS. A large number of students reported using 
three-ring binders, while a little over half reported using calendars to track work. In ACRS, students 
are often required or highly encouraged to use a three-ring binder to organize and store their class 
notes and homework and are taught a specific organizational format. The use of calendars and other 
planning and organizational tools are also strongly encouraged. 


1. Most outcome measures in Table 5.1 are the average of several survey items. Appendix B includes a list of 
all the survey items that make up each multi-item measure and the possible responses. 
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TABLE 5.1 


Student Survey Outcomes: Tenth-Grade Students, After Three Years of the AVID College 
Readiness System (ACRS) and Alignment Activities 


Estimated 

Composite Measure Fall 2013 Fall 2016 Difference P-Value 
Use of learning skills 

Frequency of use of reading and study strategies (1—4)* 2.40 2.42 0.01 0.747 

Frequency of use of critical thinking in school (1—5)° 3:57 3.49 -0.08 0.218 

Frequency of use of study habits (1-5)> 3.25 3:25 0.00 0.991 

Students report using a three-ring binder (%) 88.0 84.9 -3.1 O75 

Students report using a calendar to track work (%) 95:9 Of5 1.6 0.594 
Engagement in school* 

Tries hard and finds school interesting (1-5) 3.43 3.38 -0.04 0.376 

Focuses on long term objectives (1—5) 3.01 3.07 0.05 0.311 

Is diligent and a hard worker (1—5) 3.63 3.66 0.04 0.528 

Is confident of ability at school (1—5) Si 3.71 -0.06 0.287 
Postsecondary awareness and expectations 

Has a plan for college (1—5)° 3.83 3.79 -0.04 0.504 

Had opportunities to learn about college (0—1)¢ 0.60 0.60 0.00 0.862 

Planned level of degree attainment (1—7)° 5.14 4.99 -0.15 0.167 
School environment and academic rigor 

School focuses on success of all students (1—4)' 2.89 2.83 -0.06 0.124 

School encourages all students to go to college (1—4)' 2.75 2:16 0.01 0.858 

Environment is collaborative and respectful (1-4)! 2.84 2.78 -0.06 0.130 

Other students try hard at school (1-4) 2.9 2.62 0.04 0.424 

Students report math class was difficult but manageable (%) 38.2 45.7 74 0.016 ** 

Students report English class was difficult but manageable (%) 22.4 37.4 13.0 0.000 *** 
Number of students 2,012 


(continued) 


Engagement in School 


The second section of Table 5.1 measures students’ engagement in school, which includes measures 
of their effort and interest in school, focus on long-term objectives, diligence, and confidence in 
their school work. These measures include items such as “You try as hard as you can in school.” (See 
Appendix B for a list of the survey items included in each measure.) Again, there is little difference 
between the students who were surveyed before implementation began and those who were surveyed 
after three years of implementation. On average, students across both groups gave answers between 
“somewhat true” and “mostly true.” 
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TABLE 5.1 (continued) 


SOURCE: Surveys of students administered in fall 2013 and fall 2016. 


NOTES: Across four study high schools, 2,012 tenth-grade students participated in the survey, including 925 students in the 2013 survey 
administration and 1,087 students in the 2016 survey administration. 


The “Fall 2013” score is regression adjusted using the mean covariate values for “Fall 2016” as the basis for adjustment. 


A two-tailed t-test was used for all statistical tests presented in this table. Asterisks after the “P-Value” column represent the statistical 
significance of the difference between fall 2013 and fall 2016. Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 
percent; * = 10 percent. 


Rounding may cause slight discrepancies in calculating sums and differences. 


Data are missing for no more than 11 percent of respondents on any measure except “Planned level of degree attainment,” where data are 
missing for 15 percent of respondents. 


This measure includes several items using the following scale: 1 = rarely, 4 = almost always. 

>This measure includes several items using the following scale: 1 = never or rarely, 5 = more than once a week. 

‘This measure includes several items using the following scale: 1 = not at all, 5 = always or very much. 

‘This measure includes several items using the following scale: 0 = no, 1 = yes. 

‘This measure uses the following scale: 1 = not graduate high school, 7 = get advanced degree (e.g., master’s, doctorate). 
‘This measure includes several items using the following scale: 1 = strongly disagree, 4 = strongly agree. 


‘This measure includes several items using the following scale: 1 = none, 4 = all. 


Expectations and Awareness of Postsecondary Options 


The third section of Table 5.1 includes measures of students’ postsecondary awareness and expecta- 
tions. The two groups of surveyed students differed little in the extent to which they reported that 
they had a plan for college and had opportunities to learn about college, as well as their reported level 
of planned degree attainment. On average across both groups, students reported that it was “mostly 
true” that they had a plan for college and that they had opportunities to learn about college options 
by the start of tenth grade. While little difference was observed in the responses of all tenth-grade 
students before and after implementation, some difference was observed when the responses for the 
AVID elective students were compared before and after implementation. After the third year, AVID 
elective students were reporting that they had more opportunities to learn about college than did 
AVID elective students early in the first year of implementation. (See Appendix Table B.5.) 


School Environment and Academic Rigor 


The final section of Table 5.1 includes measures of students’ perceptions of their schools’ environ- 
ments and academic rigor, including the focus on academic success for all students, the emphasis 
on college readiness, the collaborative environment of the school, and whether students across the 
school try hard. There was little difference in the responses across the two survey groups on any 
of these measures. The first three measures in this section are scored on a scale of 1 to 4, where 
1 is “strongly disagree,” 2 is “disagree,” 3 is “agree,” and 4 is “strongly agree.” The scores average 
from 2.6 to 2.9, which means that, on average, students agreed with statements about there being a 
positive school environment and academic rigor, but there was still some room for improvement. 
While improvement was not seen across all students, AVID elective students after three years of 
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implementation were more likely to report that their school encouraged all students to go to college 
compared with AVID elective students at the beginning of the first year. (See Appendix Table B.5.) 


More students in the group responding after three years of implementation did report that their 
math and English/language arts courses were difficult but manageable as opposed to “not too dif- 
ficult,” “too easy,” or “too difficult.” This is important because AVID strives to stretch students a bit 
beyond their comfort zone in their course taking by encouraging more middle-achieving students 
to take more rigorous courses but also giving them the tools and supports they need to succeed in 
those more difficult courses. 


STUDENT ACADEMIC OUTCOMES 


To measure any changes in student academic outcomes during the implementation of the ACRS 
and alignment activities, the study team compared the administrative records and transcripts of 
students attending the four participating high schools in the three years before implementation with 
the records of students attending the four high schools during implementation.’ Table 5.2 compares 
the combined average academic outcomes of ninth- through twelfth-grade students attending the 
four participating high schools during the third year of implementation (the 2015-2016 school year) 
with the combined average academic outcomes for ninth- through twelfth-grade students attending 
the four study high schools during the three years before implementation.° 


Most students attending ninth through twelfth grades at the four study high schools during the 
third implementation year had been exposed to the ACRS during all three years of implementation. 
Since the ACRS was implemented at both the middle schools and the high schools, most ninth- and 
tenth-grade students in 2015-2016 would have experienced the system during both middle school 
and high school over the three years, and most eleventh- and twelfth-grade students would have 
experienced the ACRS in high school for all three years of implementation.‘ No version of the ACRS 
was implemented in the study’s middle schools or high schools before the first implementation 
year. Therefore, the ninth- through twelfth-grade students in the three years before implementa- 
tion represent a group of students who did not experience the ACRS. Table 5.2 compares these two 
groups of students (those with no exposure to the system and those with three years of exposure), 
but, as noted above, these two groups may also have experienced other differences in programming 
and curricula, so this study does not show whether the ACRS and alignment activities caused any 
differences found in the outcomes of the two groups. 


2. Appendix Table C.1 compares the characteristics of these two groups of students. The groups are similar, 
but there were more Hispanic students and fewer white and Asian students in the implementation years 
compared with the years before implementation. There was also a higher percentage of students with 
special education status in the third implementation year than in the years before implementation began. 
These characteristics were controlled for in the analysis. 


3. Appendix Table C.2 displays similar outcomes for students during the first year of implementation and 
Appendix Table C.3 displays outcomes for students during the second year of implementation. 


4. Students who transferred into the study schools during the three years of implementation did not 
experience the full three years of exposure to the program. 
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TABLE 5.2 


Student Outcomes in the Third Year of the AVID College Readiness System (ACRS) and 
Alignment Activities, Ninth- through Twelfth-Grade Students Combined 


Pre-AVID Year3 Estimated 

Outcome Measure Years (2015-2016) Difference P-Value 
Credits earned in advanced course work? 

Earned at least one credit in an advanced course (%) 30.1 38.1 8.0 0.000 *** 

Total advanced credits earned 0.87 0.98 0.11 0.062 * 

Core advanced credits earned? 0.74 0.88 0.13 0.004 *** 

Dual enrollment college credits earned (11th—12th grades) 0.64 0.44 -0.20 0.012 ** 

Honors and Advanced Placement credits earned (11th—12th 

grades) 0.75 0.92 017 0.012 ** 
Academic performance 

Grade point average (0—4, 9th—10th grades) 2.52 2.56 0.03 0.380 

State English Language Arts achievement level (1-5, 

3 = passing, 9th—10th grades) 2.25 220 -0.01 0.837 
Educational attainment 

Total credits earned 5.15 5.11 -0.04 0.580 

Core credits earned 3:27 O21 0.01 0.922 

Promotion to the next grade (%) t19 80.5 2.6 0.135 

High school graduation (%) 85.7 83.2 -2.5 0.383 
High school persistence 

Retention in school through the school year (%) 88.6 88.4 -0.2 0.847 
Sample size 17,669 


SOURCE: MDRC calculations are based on student records obtained from school districts for 2010-2011 through 2012-2013 (pre-AVID 
years) and 2015-2016 (Year 3). 


NOTES: Across four study high schools, 12,942 students were included in the pre-AVID implementation sample, which includes students in 
the study high schools during the three years before AVID implementation, and 4,727 students were included in the post sample, which includes 
all students in the study high schools during the 2015-2016 school year. 


The “Pre-AVID Years” column displays the regression-adjusted average scores for all students in ninth through twelfth grades during 
the three years before the first year of implementation, and the “Year 3” column displays the average scores at the end of the third year of 
implementation of the ACRS and Alignment Activities. 


A two-tailed t-test was used for all statistical tests presented in this table. Asterisks after the “P-Value” column represent the statistical 
significance of the difference between Year 3 and the pre-AVID years. Statistical significance levels are indicated as follows: *** = 1 percent; ** 
= 5 percent; * = 10 percent. 


Rounding may cause slight discrepancies in calculating sums and differences. 


Data are missing for no more than 4 percent of respondents on any measure except for the state English Language Arts achievement level. 
On that measure, 14 percent of the data are missing for the students after three years of implementation, and 21 percent of the data are missing 
for the pre-AVID implementation group. Also, in the pre-AVID implementation group, most data were missing for tenth-grade students in one 
district during 2010-2011 and 2011-2012, so only 2012-2013 data were included for that district and grade. 


One district did not provide data on dual enrollment courses taken and passed. For this district, dual enrollment courses are not included in 
the measures of advanced credits earned, and the district is not included in the measure of dual enrollment credits earned. 


>Core courses include all English/language arts, math, science, and social studies courses. 


°Starting in spring 2015, the Florida state assessment changed from the Florida Comprehensive Assessment Test to the Florida Standards 
Assessment. The achievement levels for these tests are not equivalent, and some caution should be taken when interpreting these findings. 
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Credits in Advanced Course Work 


The first section of Table 5.2 displays student outcomes on enrollment and credits earned in honors, 
dual enrollment, and Advanced Placement courses. According to the AVID philosophy, ensuring 
that a broader cross-section of students take more advanced and rigorous courses, and offering those 
students additional support to succeed in those courses, will lead to higher levels of college readi- 
ness across the school. Approximately 8 percent more students earned at least one advanced credit 
during the third year of implementation compared with the years before implementation, and, on 
average, students in the third year also earned slightly more credits in advanced courses than did 
students in the years before implementation.°® 


Students also attempted more advanced credits during the implementation year than in the years 
before implementation. This finding suggests that, on average, students were taking more advanced 
courses and succeeding in those courses. Although it is not possible in this analysis to determine 
the cause of the change in rigorous course taking and success, the findings match with a key goal of 
ACRS to first push students to stretch themselves by taking more difficult courses, and at the same 
time, to ensure that those students have a strong foundation in learning strategies and study habits 
to help them succeed in those tougher classes. 


Dual enrollment courses are offered in many different subject areas sponsored by the local college. 
The courses, which can be offered at the high school or on the college campus and are taught by 
high school or college teachers, give students the opportunity to earn both high school and college 
credits simultaneously. The ACRS is intended to encourage students to take more of those courses, 
and one of the goals of the alignment activities in the AVID study was to build collaboration between 
the high schools and the college. Still, eleventh- and twelfth-grade students earned fewer dual en- 
rollment college credits during the third year of implementation than did students during the years 
before implementation. One probable reason for this decline in dual enrollment may have been a 
lack of resources at some schools, including a lack of certified teachers to cover the dual enrollment 
courses. When fewer teachers at the high schools are able to take responsibility for these courses, 
students need to take the courses at the college campus, but associated schedule and transportation 
limitations make it difficult for students to do this.® 


Although students were earning fewer dual enrollment credits in the third year, on average, those 
courses were supplanted by Advanced Placement and honors courses.’ Eleventh- and twelfth-grade 
students earned more honors and Advanced Placement credits during the third implementation year 
than students in those grades during the years before implementation. A past study has shown that 


5. Although there is a positive difference in advanced credit earning between the third year of implementation 
and the average of the three years before implementation, advanced credit earning did steadily increase 
during the three years before program implementation. The change in advanced credit earning during the 
program years could be due to this steady increase and not be connected to implementation. 


6. As shown in Appendix Table C.5, one of the high schools shows a particularly large drop in dual enrollment 
credit earning during the third program year, while having a higher level of dual enrollment credit earning in 
the pre-AVID implementation years than the other schools. This finding may be fueled in part by changes 
in dual enrollment offerings at that high school over the study years. 


7. Advanced Placement courses are courses taken at the high school that prepare students for Advanced 
Placement exams. If students take and earn passing scores on the exams, they can earn college credit. 
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taking more Advanced Placement courses and exams is related to higher scores on college entrance 
exams and higher matriculation into college.® 


Academic Performance 


The second section of Table 5.2 displays students’ academic achievement by measuring their average 
grade point average (GPA) across all high school grades, and ninth- and tenth-grade students’ average 
English/language arts achievement level on the Florida State English Language Arts assessment. GPA 
is measured on a 4.0 scale, where 4 is an A average and 0 is failing. The state assessments are scored 
on a scale of 1 to 5, with 5 the highest score a student can achieve. Students scoring a 3 or above 
meet the state proficiency level. The Florida State English Language Arts assessment changed from 
the Florida Comprehensive Assessment Test (FCAT and FCAT 2.0) in the pre-implementation years 
to the Florida Standards Assessment in English Language Arts in the third year of implementation. 
The standard for proficiency is different across the tests. 


The average outcomes for students on these measures are similar in the third year of implementation 
to the average of the outcomes during the three years before implementation, suggesting that there 
was little change in English Language Arts state test performance and GPA after the AVID program 
was implemented. As noted above, the state test changed during the pre- and post-implementation 
years, and the proficiency levels of the two tests are not directly comparable. A 2.56 GPA is equivalent 
to a B-minus or C-plus average across classes. This is an unweighted GPA, meaning that honors and 
other advanced-level courses are not given additional points. Even though students were earning 
at least one more advanced-level credit during the third year of implementation than in the years 
before implementation, they were not getting lower grades. The fact that the average GPA across the 
schools is maintained while students were taking more advanced courses on average may suggest 
some growth in students’ academic performance during implementation. 


Educational Attainment 


The third section of Table 5.2 displays the outcomes for students’ overall educational attainment, 
including their total credits earned during the school year, their core course credits earned dur- 
ing the school year (core courses include English/language arts, math, science, and social studies), 
whether they were promoted to the next grade at the end of the school year, and, for twelfth-grade 
students, whether they graduated and received a high school diploma. Little difference is seen be- 
tween the third year of implementation and the years before implementation on these measures. 
Although there is little difference in promotion seen for all grades averaged together, there is almost 
a 10 percentage point difference for ninth-grade students (as shown in Appendix Table C.4) — that 
is, after three years of AVID implementation, ninth-graders were 10 percentage points more likely 
to be promoted to tenth grade than were those in the pre-implementation years. This difference 
(which is statistically significant) is important because ninth grade is a transition year and is often 
the time when struggling students fall behind. 


8. Jackson (2010). 
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High School Persistence 


The final section of Table 5.2 displays students’ persistence in high school during the school year. 
Although some students drop out of school or leave the district during each school year, the rate of 
persistence in school, approximately 88 percent, remains about the same in the third year of imple- 
mentation as it was in the pre-implementation years. 


Eighth-Grade Students 


Table 5.3 displays similar outcomes for eighth-grade students, including their credits earned in 
honors classes, their GPA, their achievement level on the Florida State Reading test, their total and 
core credits earned, and their promotion to the next grade. Because the ACRS was rolled out to only 
seventh- and eighth-grade students in the middle schools during the first year of implementation 
(2013-2014), eighth-grade students in the third year of implementation (2015-2016) had experienced 
the ACRS for only two years (in seventh grade in 2014-2015 and in eighth grade in 2015-2016). 


Similar to the high school findings, eighth-grade students, on average, took and succeeded in more 
advanced courses during implementation than in the years before implementation. Fifteen percent 
more students earned at least one honors credit in the third implementation year and students in 
the third year earned more honors course credits, on average, than did their counterparts in the 
years before the study. 


There was little difference in academic performance (students’ average GPA and reading achievement 
level) or educational attainment (total and core credits earned and promotion to the next grade) 
between the third implementation year and the prior years. 


Postsecondary Outcomes 


Table 5.4 displays the percentage of high school graduates who enrolled in any public or private U.S. 
college in the fall directly after graduation from each school and overall. The first section shows 
the results for students who graduated in 2016. These are the twelfth-grade students who had three 
years of exposure to the ACRS — in grades 10, 11, and 12 — if they attended the school for all three 
years. The second and third sections of the table display the outcomes for students who graduated 
during the prior two years, 2015 and 2014. Although the system was being implemented during these 
years, these students were in grades that were not targeted for ACRS implementation (eleventh and 
twelfth grades were not targeted in the 2013-2014 school year, and twelfth grade was not targeted 
in the 2014-2015 school year) and so only received limited exposure to program components that 
may have filtered across the grade levels. For instance, these students may have taken some classes 
with teachers who had been trained in the WICOR model, but those strategies were not being imple- 
mented across their classes as they were with the targeted grades. The districts in the study started 
collecting postsecondary data as part of the implementation study, and the data are not available 
for the years before implementation. Each high school showed some fluctuation in postsecondary 
enrollment over the three years, but on average across the high schools, approximately 50 percent of 
high school graduates enrolled directly into college during each of the three implementation years, 
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TABLE 5.3 


Student Outcomes in the Third Year of the AVID College Readiness System (ACRS) and 
Alignment Activities, Eighth-Grade Students 


Pre-AVID Year3 Estimated 

Outcome Measure Years (2015-2016) Difference P-Value 
Credits earned in honors classes 

Earned at least 1 credit in an honors course (%) 19.7 30,1 15.4 0.002 *** 

Total honors credits earned 0.40 0.81 0.41 0.016 ** 

Core honors credits earned? 0.42 0.75 0.33 0.017 ** 
Academic performance 

Grade point average (O—4) 2.93 2.90 -0.03 0.720 

State English Language Arts achievement level’ (1-5, 

3 = passing) 2.26 2.40 0.14 0.259 
Educational attainment 

Total credits earned 4.53 4.99 0.46 0.391 

Core credits earned? 3.45 3.48 0.03 0.890 

Promotion to the next grade (%) 94.9 91.8 -3.1 0.102 
Sample size 4,839 


SOURCE: MDRC calculations are based on student records obtained from school districts for 2010-2011 through 2012-2013 (pre-AVID years) and 
2015-2016 (Year 3). 


NOTES: Across the four study middle schools, 3,613 students were included in the pre-AVID implementation sample, which includes eighth- 
grade students in the study middle schools during the three years before AVID implementation, and 1,226 students were included in the Year 3 
implementation sample, which includes all students in the study middle schools during the 2015-2016 school year. 


The “Pre-AVID Years” column displays the average scores for all students in ninth through twelfth grades during the three years before the first 
year of implementation, and the “Year 3” column displays the average scores at the end of the third year of implementation of the ACRS and alignment 
activities. 


A two-tailed t-test was used for all statistical tests presented in this table. Asterisks after the “P-Value” column represent the statistical significance 
of the difference between Year 3 and pre-AVID years. Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; * = 10 
percent. 


Rounding may cause slight discrepancies in calculating sums and differences. 


Data are missing for no more than 5 percent of respondents on any measure except for the the state English Language Arts achievement level. On 
that measure, 13 percent of the data are missing for the students after three years of implementation, and 8 percent of the data are missing for the 
pre-AVID implementation group. 


Core courses include all English/language arts, math, science, and social studies courses. 


One middle school is not included in this analysis due to missing data. Starting in spring 2015, the Florida state assessment changed from the 
Florida Comprehensive Assessment Test to the Florida Standards Assessment. The achievement levels for these tests are not equivalent, and some 
caution should be taken when interpreting these findings. 
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TABLE 5.4 
Enrollment in College in the Fall Following High School 
Graduation: 2014, 2015, and 2016 High School Graduates 


High School 

Graduate Year A B Cc D_ Total 
2016 graduates 

High school graduates (N) 216 166 212 257 851 

Enrolled in postsecondary institution (%) 52.3 51:2. B87 55.6 49.7 
2015 graduates 

High school graduates (N) 179 161 194 237 (fl 

Enrolled in postsecondary institution (%) 58.1 54.7 33.0 565 50.6 
2014 graduates 

High school graduates (N) 205 160 172 250 787 


Enrolled in postsecondary institution (%) 49.8 58.1 44.2 51.6 . 50.8 
SOURCE: National Student Clearinghouse reports for each school district for 2014, 2015, and 2016. 


and little difference in the percentage can be seen between the years. This level of college enrollment 
is below the Florida state average, which was approximately 62 percent in 2015.’ 


CONCLUSION 


Although little difference was found in reported study habits, engagement in school, and college 
aspirations between the group of tenth-graders before implementation and the tenth-graders after the 
third year of implementation, students who had participated in the ACRS were more likely than their 
counterparts before implementation to report that their math and English/language arts classes were 
difficult but manageable as opposed to too easy or too difficult. At the same time, more high school 
students were taking an advanced course and 8 percent more were earning at least one credit in an 
advanced course in the third year than in the years before implementation, and the average number 
of credits earned across all students was also higher. Although it cannot be determined whether the 
ACRS and alignment activities caused this positive change in advanced credit earning, a key goal 
of the ACRS is to ensure that more students take advanced courses and have the support needed to 
succeed in those courses. While, on average, students’ advanced credits earning grew during the 
implementation years, other measures of their academic performance, educational attainment, and 
high school persistence remained similar. Still, the fact that students were taking and succeeding 
in more advanced courses suggests that they were likely becoming better prepared for college-level 
work by the time of high school graduation. Postsecondary enrollment for high school graduates 
fluctuated across high schools, but remained similar across all four high schools during the study, 
with approximately 50 percent of students enrolling directly into college in the fall after high school 
graduation. While the growth in advanced credits earning is important for adequate college prepara- 
tion, further work may be needed to raise students’ college aspirations and enrollment. 


9. Florida Education and Training Placement Information Program (2017). 
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Conclusion and Reflections 


he AVID Central Florida Collaborative is a partnership between the AVID Center and key 

stakeholders in secondary and postsecondary education across a rural area in central Florida 

that was created in an effort to build college readiness and tackle low college enrollment 

and completion. To achieve this goal, the AVID College Readiness System (ACRS) was 
implemented in four sets of middle schools and high schools across three school districts. At the 
same time, the school districts, schools, and local state college piloted a set of alignment activities 
that created avenues for collaboration across grade levels, schools, districts, and content areas, and 
between the local middle schools and high schools and the postsecondary institution. This chapter 
summarizes the study findings and offers some reflections on implementing the program and build- 
ing on the program’s successes. 


SUMMARY OF FINDINGS 


The ACRS components (professional development, the AVID elective, and the site team) were im- 
plemented with more success more quickly than the alignment activities, in all likelihood because 
although the ACRS components were new to the participating schools, the AVID Center has been 
developing and implementing them in schools across the country for many years. Professional de- 
velopment was implemented with the most success. The AVID elective was also successful despite 
some schools’ struggles throughout the three years with recruiting and keeping college student tutors 
for the tutorial component of the elective. The site teams took longer to establish in several of the 
schools, but they were successfully implemented in most schools by the third year. 


The alignment activities — middle school and high school feeder teams, teacher content collabora- 
tives, the cross-district vertical articulation collaborative, and the state college partnership — were 
new features designed specifically for this project. Not surprisingly, they were often more difficult to 
implement and took longer to establish than the ACRS components. Middle and high school feeder 
teams met sporadically in the first two years and most of them did not have actionable plans even 
in the third year. The teacher content collaboratives also struggled to get off the ground, suffering 
from a lack of definition and focus, as well as attendance challenges, but were more successful by 
the third year. The vertical articulation collaborative, through which district and school administra- 
tors met and created a productive community of practice, was generally implemented with fidelity 
to the program model. While the college enthusiastically embraced the WICOR (writing, inquiry, 
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collaboration, organization, and reading) model, the teacher content collaboratives and vertical 
articulation collaborative were poorly attended by college faculty and administrators. 


The successful implementation of the ACRS components and the strong community of practice cre- 
ated across districts and secondary schools through the vertical articulation collaborative meant that 
the schools and districts did embrace some of the new programming despite the obstacles and the 
incomplete implementation of the alignment activities. The growth of the ACRS across the schools 
likely contributed to the positive changes found in teachers’ reported use of the AVID teaching meth- 
odologies and strategies, in school staff members’ expectations of students, and in collaboration by 
school staff within and across schools as well as across secondary and postsecondary institutions. 
Regardless of the specific causes of these changes, the findings suggest that the AVID methodolo- 
gies and philosophy did begin to take hold across the schools during the three years of the study. 


A comparison of students’ reported use of educational strategies, of their engagement in school, and 
of their postsecondary expectations before implementation and after three years of implementation 
showed little change in students’ reported attitudes, actions, and expectations, but student responses 
were relatively high for both groups on many of the measures. Students took more advanced courses 
during the implementation years compared with the years before implementation. There was growth 
in students’ credit earning in advanced courses, including honors and Advanced Placement, with 
8 percent more high school students earning at least one advanced course credit in the third year 
of implementation compared with the years before implementation. Although this finding cannot 
be directly attributed to the program, this change in advanced credit earning was a key program 
goal. Earning more credits in advanced courses is closely related to some successfully implemented 
program components, including professional development and the AVID elective. Other student 
outcomes, including academic performance, educational attainment, and school persistence, did not 
change substantially. Likewise, little change was observed in the number of students who enrolled in 
college directly after high school, but it is possible that more time is needed to affect these outcomes. 


ACADEMIC RIGOR 


There are some interesting connections between certain study findings reported by school staff and 
students and found in the analysis of student records. These connections convey a possible story 
about how the ACRS and alignment activities may have taken hold at the schools, affected the school 
culture, and led to changes in students’ outcomes. Teachers and other school staff reported growth 
in the schools’ recognition of students’ academic achievement, their shared understanding of what 
students should know in each grade level, and the efforts of students to do well in school. Staff 
members at the schools during the implementation years were also more likely to report feeling that 
all students at the school could be successful in college in comparison with staff members’ beliefs 
during the pre-implementation years. Teachers also reported a wider use of many of the WICOR 
model strategies meant to build academic rigor in the classroom and to support students’ study and 
learning habits during the implementation years. 


These changes in teacher practice and in school staff members’ expectations for students could be 
connected to an increased likelihood among students after experiencing the program to report that 
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their math and English/language arts classes were “difficult but manageable” as opposed to “too 
easy” or “too difficult.” This finding also seems closely related to the finding that students were tak- 
ing and passing more advanced courses during the ACRS implementation than before it. All these 
findings point to a culture change across the staff at the schools toward a stronger belief that more 
students can succeed in rigorous course work and attend college, and toward ensuring that more 
students are given the opportunity to take rigorous college-track courses. 


COMMUNITIES OF PRACTICE FOR TEACHERS 


Research suggests that teacher communities of practice are key to students’ successful transitions 
through school and their postsecondary education success.' These communities provide a forum for 
content area teachers across grades and across secondary and postsecondary institutions to come 
together to align their curricula and teaching strategies across grade levels and to work together 
to clearly identify expectations for students in each grade. Establishing this type of community is 
difficult anywhere, as teachers’ busy days allow little time for any additional effort, and it is particu- 
larly difficult to connect secondary and postsecondary educators given that they work in separate 
institutions that allow for little interaction.? These hurdles can be exacerbated in a rural area where 
geographic distance and the smaller teaching staff at schools can play a large role in isolating teach- 
ers from this kind of community. 


The AVID Central Florida Collaborative struggled with these issues in creating a successful com- 
munity of practice through the teacher content collaboratives. One of the biggest challenges was 
finding enough substitute teachers to fill in when teachers attended content collaboratives, which 
were scheduled during the school day. After-school meetings were also attempted, but teachers did 
not have the time or the energy to devote to these meetings after teaching all day. Rural secondary 
and postsecondary schools that are interested in this type of collaboration may need to identify a 
realistic time for teachers to attend and offer compensation for the additional effort in order to en- 
sure strong teacher participation, which is probably necessary for this kind of effort to be effective. 


Beyond ensuring that kindergarten through twelfth grade and postsecondary faculty attend, much 
structure and guidance may be needed, at least during the first few meetings, to set a standard for 
success so that the value of these meetings is immediately evident to participating teachers. Including 
more school and district leadership and district-level content and curriculum specialists may have 
been useful in fostering structure and goal setting. School and district leadership may need to be 
involved in initiatives like the content collaborative, the site teams, and the middle school and high 
school feeder teams to identify the teachers and other staff members who are most able to take on 
these tasks, to support effective implementation, and to ensure alignment across all school and 
district leadership. 


1. McRobbie (2004). 
2. Barnett et al. (2012). 
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DUAL ENROLLMENT VERSUS ADVANCED PLACEMENT 
COURSES 


Across the high schools in the study, dual enrollment credit earning for eleventh- and twelfth-grade 
students declined during the implementation years, while other advanced course taking (honors and 
Advanced Placement courses) grew during the implementation of the program. A school’s decision 
to emphasize Advanced Placement, honors, or dual enrollment courses to reach the goal of better 
preparing students for college may be based on a number of factors. Dual enrollment courses may 
offer students a chance to acquire some college credits so they have a head start on college comple- 
tion, but taking and passing an Advanced Placement test is another option toward earning college 
credit during high school. 


Anecdotally, some high school principals said they preferred offering more Advanced Placement 
courses than dual enrollment courses at their schools because they believe that Advanced Placement 
courses are “more indicative of the rigorous instruction of a college class.” Still, passing a dual enroll- 
ment course leads directly to college credit earning at the sponsoring college, while students need to 
take and pass an additional test to earn college credits through Advanced Placement courses. High 
school and secondary-postsecondary collaborative partnerships should pay attention to how their 
students are best able to attain college credits (through dual enrollment or Advanced Placement 
courses or a mixture of both), and also try to compare the rigor of these different types of courses 
to be as thoughtful as possible about which offerings for students lead to the best college preparation 
and college success for the most students. 


COLLEGE COLLABORATION 


While state college faculty did not fully participate in the teacher content collaboratives and the ver- 
tical articulation collaborative, enthusiasm for the WICOR model was strong across the institution. 
The use of the WICOR model by a local postsecondary institution could encourage and strengthen 
the use of these methodologies, along with the AVID core values, across grade levels at secondary 
institutions. That is, if middle school and high school teachers know and understand that many of 
their students will be expected to use the WICOR model strategies and will be exposed to these 
teaching methods during college, it may promote their use in secondary school and help to build a 
continuum of rigorous teaching methods and learning strategies across secondary and postsecondary 
education in a community. The use of shared language and a set of methodologies across secondary 
and postsecondary educators could also allow for more effective future dialogues between these 
educators. Moreover, the consistency of methods across grades and schools could lead to longer-term 
success by helping to ease the transition into postsecondary education for these students. 
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APPENDIX 


Implementation Research Supplemental 
Materials 


APPENDIX TABLE A.1 


AVID College Readiness System (ACRS) and Alignment Activities Fidelity Matrix 


Key Component or Element 


Operational Definition 


Data Source(s) 


ACRS 
Professional development 
Summer Institute 


Path to Schoolwide 


AVID elective teacher and AVID 
coordinator coaching 


AVID elective 


Class offering 


Recruitment and enrollment 


Organizational skills 


Inquiry and collaboration 


Reading and writing 


Tutoring 


Site team 


15 staff members per school participate in 3 
full days of training in summer 2013; 8 staff 
members per school participate in 3 full days 
of training in summer 2014 and 2015. 


10 staff members per school participate in 
2 full days of training in spring 2014, 2015, 
and 2016. 


At least 1 AVID elective teacher and 
the AVID coordinator from each school 
participate in 1 coaching meeting per 
semester in school years 2013-2014, 
2014-2015, and 2015-2016. 


Year-long AVID elective classes are held 
during the regular school day, taught by 
trained AVID teachers. 


Prescribed recruitment and enrollment 
strategies are used; enrollment in the AVID 
elective classes meets AVID criteria. 


Instructional strategies are taught in AVID 
elective classes to develop organizational 
skills that promote academic self- 
management. 


Inquiry and collaboration are used as a 
basis for instruction in the AVID elective 
classes. 


A strong, relevant writing and reading 
curriculum provides a basis for instruction in 
the AVID elective classes. 


Students in the AVID elective classes 
receive support from trained tutors. 


An active interdisciplinary AVID site team 
collaborates on issues of student access to 
and success in rigorous college preparatory 
courses. 


Attendance rosters 


Attendance rosters 


Attendance rosters 


AVID certification form 


AVID certification form 


AVID certification form 


AVID certification form 


AVID certification form 


AVID certification form 


AVID certification form 
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(continued) 


APPENDIX TABLE A.1 (continued) 


Key Component or Element 


Operational Definition 


Data Source(s) 


Alignment activities 


Middle school—high school 


feeder teams 
Meeting attendance 


Plans 


Teacher content collaboratives 


Facilitation 


Meeting attendance 


Plans 


Vertical articulation collaborative 


(VAC) 
Facilitation 


Meeting attendance 


Plans 


State college partnership 
AVID training 


Participation in teacher content 


collaboratives 


The principals and AVID coordinators from 
a middle school—high school pair meet for 1 
hour, once per year, in school years 2013-— 
2014, 2014-2015, and 2015-2016. 


Feeder teams develop written plans, which 
include specific action items to support the 
implementation of the ACRS at the schools. 


AVID convenes and facilitates two full-day 
meetings per year for each content area 
(English/language arts, math, science, and 
social studies) in school years 2013-2014, 
2014-2015, and 2015-2016. 


At least 2 content area teachers per school 
attend each meeting. 


Written plans are developed each year 
addressing alignment of curricula across 
grades, student engagement, and use of the 
WICOR model.? 


AVID convenes and facilitates VAC meetings 
3 times per year for a full day in school years 
2013-2014, 2014-2015, and 2015-2016. 


8 principals, 1 representative from Heartland 
Educational Consortium, 1 representative 
from South Florida State College, and the 

3 district directors (13 people) attend each 
meeting. 


The VAC develops written plans addressing 
calibration of rigor, leadership, student 
engagement, and the implementation of the 
WICOR model.? 


20 state college staff members participate 
in 3 full days of training in school years 
2013-2014, 2014-2015, and 2015-2016. 


2 state college instructors per content area 
participate in teacher content collaboratives. 


Attendance rosters 


Document review 


Document review 


Attendance rosters 


Document review 


Document review 


Attendance rosters 


Document review 


Attendance rosters 


Attendance rosters 


(continued) 
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APPENDIX TABLE A.1 (continued) 


SOURCE: AVID i3 Implementation Fidelity Matrix. 


NOTES: Implementation of each program element was measured against criteria established in advance. The level of 
implementation fidelity for each element was based on these criteria (for example, the degree to which actual rates of participation 
in meetings and professional development sessions matched desired rates of participation). 


‘The WICOR model includes teaching strategies in writing, inquiry, collaboration, organization, and reading. 
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APPENDIX 


Staff and Student Survey Supplemental 
Materials 


This appendix includes supplementary materials connected to the staff and student surveys discussed 
in Chapters 4 and 5. The first section describes the response rates for each of the surveys in each of 
the years they were administered. The second section describes the creation of the measures used 
in the analyses where more than one survey item is combined. The final section provides some ad- 
ditional analyses of staff and student surveys. 


STAFF AND STUDENT SURVEY RESPONSE RATES 


Appendix Table B.1 describes the response rates and sample sizes for the staff surveys. Findings 
from the analyses of data from the staff survey can be found in Chapter 4, Table 4.1. The survey was 
administered during the spring before program implementation (2013) and in each spring for the 
three years of program implementation that are included in this evaluation (2014, 2015, and 2016). 
Appendeix Table B.1 shows the response rate for teachers, administrators, and guidance counselors 
separately, as well as the combined response rates for all three. Response rates were 70 percent or 
above for all types of respondents. 


Appendix Table B.2 displays the response rates and sample sizes for the student survey that was 
administered to all tenth-grade students in the early fall of the first year of implementation (2013), 
before students had much exposure to the program, and again to a new set of tenth-grade students in 
the early fall after the third year of implementation (2016) when tenth-grade students had experienced 
the program for three years in both middle school and high school. Findings from the analyses of 
the student survey data can be found in Chapter 5, Table 5.1. Appendix Table B.2 shows the response 
rates for the two survey administrations at each of the four high schools. As can be seen from the 
table, response rates were 70 percent or higher for each group of respondents. 


STAFF AND STUDENT SURVEY MEASURE CREATION AND 
FACTOR ANALYSIS 


Many of the school and students outcomes that were measured using data from the staff and student 
surveys (found in Chapters 4 and 5, Tables 4.1 and 5.1, respectively) were created by combining more 
than one survey item. The descriptions on the following pages include each constructed survey item 
and explain the creation of the measure, the survey items included, the scale used, and the Cronbach’s 
alpha from the factor analysis. 


64 | Building College Readiness Across Rural Communities 


APPENDIX TABLE B.1 
Staff Survey Response Rates 


Guidance 
Time of Survey Teachers Administrators Counselors Total 
Pre-AVID year 
Response rate (%) 82.5 71.4 89.5 82.3 
Sample size 479 21 19 519 
Year 1 (2013-2014) 
Response rate (%) 81.8 90.9 100.0 82.8 
Sample size 483 22 19 524 
Year 2 (2014-2015) 
Response rate (%) 81.5 14.1 90.5 81.5 
Sample size 486 27 21 534 
Year 3 (2015-2016) 
Response rate (%) 79.2 85.7 94.7 80.0 
Sample size 486 21 19 526 
APPENDIX TABLE B.2 
Student Survey Response Rates 
High School 
Time of Survey A B Cc D 
Fall 2013 
Response rate (%) 70.8 70.4 100.0 93.6 
Sample size 250 186 293 346 
Fall 2016 
Response rate (%) 91.9 87.1 82.1 94.8 
Sample size 270 232 312 402 
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SCHOOL OUTCOMES 


WICOR Model 


Writing. The value for this construct was determined by calculating the average of the survey item 
responses (five items, Cronbach’s alpha = 0.84). 


How often during a term did you do the following? 


e Ask students to revise their notes and/or create a summary of their notes (from readings, class- 
room lectures, etc.). 


e Ask students to write about what they have read, reflecting on a section, chapter, or unit. 
e Ask students to write a summary sentence in order to synthesize a passage. 


e Ask students to write in journals or logs reflecting on what they have been learning in their classes, 
as well as how they are doing and/or what goals they have for themselves. 


e Provide your students with class time to write to a prompt that is tied to your curriculum. 


Scale: 1 = never, 2 = once per term, 3 = at least once a month, 4 = more than once a month but not 
weekly, 5 = once a week, 6 = more than once a week but not daily, and 7 = daily 


Inquiry. The value for this construct was determined by calculating the average of the survey item 
responses (four items, Cronbach’s alpha = 0.65). 


How often in a term did you do the following? 


e Ask students to agree/disagree with a prompt where one student speaks at a time going back and 
forth from the pro to the con. 


e Ask students to work in small groups, asking each other questions about the subject matter or 
texts to discover answers to questions as a group. 


e Spend time helping students learn the meaning of new words. 


e Ask students to apply what they have previously learned to do what they are currently doing in 
the classroom. 


Scale: 1 = never, 2 = once per term, 3 = at least once a month, 4 = more than once a month but not 
weekly, 5 = once a week, 6 = more than once a week but not daily, and 7 = daily 
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Collaboration. The value for this construct was determined by calculating the average of the survey 
item responses (six items, Cronbach’s alpha = 0.8178). 


How often in a term did you do the following? 


Ask students to work in small groups on a product with a rubric of expectations. 
Ask students to debate a statement or question in written form only, utilizing chart paper. 


Showcase student work by posting it around the room and then moving in small groups from 
example to example discussing the work with them. 


Ask students in a group to each read a separate “chunk” of text, summarize what they read to 
others in their group who read different chunks, and then work together to learn the entire mean- 


ing of the text. 


Ask students to think about a question prompt on their own, and then discuss it with a classmate 
or classmates. 


Ask students to respond to a prompt through dialogues with the student sitting next to them. 


Scale: 1 = never, 2 = once per term, 3 = at least once a month, 4 = more than once a month but not 
weekly, 5 = once a week, 6 = more than once a week but not daily, and 7 = daily 


Organization. The value for this construct was determined by calculating the average of the survey 
item responses (eight items, Cronbach’s alpha = 0.75). 


How often in a term did you do the following organizational tools in your classroom? 


Three ring binders to keep work in and keep it orderly. 
Assignment logs to record work and grades on that work. 


Agendas or calendars were used in the classroom to record due dates, homework and expecta- 
tions on assignments. 


Spiral notebooks for recording notes in an interactive format (i.e., lecture notes, notes and hand- 
outs on one side, and student-generated work on the other). 


Notes chunked into three categories/columns of questions, facts, and steps. 


Rubrics or other clear guidelines to explain expectations for assignments including the point value 
of specific components, which is given to the students when an assignment is made. 


Use of “foldables” that is folding paper to help students organize and record information into 
categories. 
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Essay planning where students first formulate and state a clear thesis and organize details and 
facts to support that thesis writing prior to writing. 


A form for writing assignments to assist students with organizing the facts/details to use, identify 
a thesis statement, etc. 


Scale: 1 = never, 2 = once per term, 3 = at least once a month, 4 = more than once a month but not 


weekly, 5 = once a week, 6 = more than once a week but not daily, and 7 = daily 


Reading. The value for this construct was determined by calculating the average of the survey item 


responses (eight items, Cronbach’s alpha = 0.89). 


How often during a term did you do the following? 


Ask students to read complex texts in the classroom. 


Have students number the paragraphs, circle key terms, underline author’s claims, and use this 
information to engage in activities about the text. 


Use guided reading techniques that assist students in determining the meaning of the passage 
and the author’s purpose. 


Employ close reading techniques that allow for the students to repeat and/or fill in the blanks as 
the class reads together. 


Use rereading techniques that require students to read a passage more than once, with a different 
focus each time, to ensure comprehension. 


Have students participate in Socratic seminars—that is, engage in collaborative dialogue about 
the text through the use of higher level questioning. 


Ask students to summarize texts, pulling out the most important information in a concise wrap up. 


Ask students to use tables, graphs, or pictures to organize the information in the text into a more 
understandable form (such as Venn Diagrams, Acrostics, Spider Diagrams, Timelines, etc.). 


Scale: 1 = never, 2 = once per term, 3 = at least once a month, 4 = more than once a month but not 


weekly, 5 = once a week, 6 = more than once a week but not daily, and 7 = daily 


AVID Core Values 


Academic rigor. The value for this construct was determined by calculating the average of the survey 


item responses (three items, Cronbach’s alpha = 0.76). 
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How strongly do you disagree or agree with the following statements about your school and students? 


e Your school recognizes and acknowledges the academic achievement of its students. 
e Your schools values students’ academic achievement. 


e Teachers, administrators, and staff at your school have a shared understanding of what each 
student should know when they enter and leave each grade level at your school. 


e Students at your school respect other students who work hard to get good grades. 
e Students at your school try to get good grades. 
Scale: 1 = strongly disagree, 2 = somewhat disagree, 3 = somewhat agree, and 4 = strongly agree 


Collaboration and Vertical Alignment 


Meetings within and across schools. The value for this construct was determined by calculating 
the average of the survey item responses (four items, Cronbach’s alpha = 0.79). 


How often during a month did you do the following? 


e Met with colleagues at your school during a designated time. 
e Met with colleagues from other schools during a designated time. 
e Met with colleagues to discuss the needs of specific students. 


e Met with colleagues to discuss issues of student access and success in rigorous college prepara- 
tory courses? 


e Participated in meetings with colleagues within your school to align teaching methodologies, 
expectations, and/or curriculum across grade levels. 


e Participated in meeting with colleagues from both middle and high schools in your district to 
align teaching methodologies and/or curriculum across grade levels. 


e Met with postsecondary teachers in your area to align teaching methodologies and/or curriculum 
across grade levels. 


Scale: 1 = never, 2 = at least once a month, 3 = more than once a month but not weekly, 4 = once a 
week, 5 = more than once a week but not daily, and 6 = daily 


Building College Readiness Across Rural Communities | 69 


Student Outcomes 


Use of Learning Skills 


Frequency of use of reading and study strategies. The value of this construct was determined by 
calculating the average of the survey item responses (eight items, Cronbach’s alpha = 0.90). 


Please tell us how frequently you do each of the following. 


e As you read, you stop sometimes and mentally summarize what you have read and put it into 
your own words. 


e You underline or make notes on the important information from a reading assignment. 


e You test your understanding by trying to explain the information in your own words as you review 
the material you have read. 


e You organize notes from class into main ideas and details or examples. 


e As you review your notes, you test your understanding by trying to explain information in your 
own words. 


e When you have a lot of material to remember, you break material into parts before attempting 
to learn it. 


e You create diagrams or pictures to help you remember information. 


You test yourself sometimes to practice remembering material you have studied. 
Scale: 1 = rarely, 2 = sometimes, 3 = frequently, 4 = almost always 


Frequency of use of critical thinking in school. The value of this construct was determined by 
calculating the average of the survey item responses (four items, Cronbach’s alpha = 0.81). 


In general, about how often do you... 


e Solve problems in your classes or for homework that require you to think for yourself? 
e Work on assignments or projects that require you to think creatively in school? 
e Learn about or discuss strategies for time management and how to stay organized in school? 


e Get asked to apply what you have previously learned to what you are currently doing in your 
classes or your homework? 


Scale: 1 = never or rarely, 2 = occasionally (less than once a month), 3 = once or twice a month, 4 = 
several times a month to weekly, 5 = more than once a week 
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Frequency of use of study habits. The value of this construct was determined by calculating the 


average of the survey item responses (seven items, Cronbach’s alpha = 0.88). 


How often do you... 


Revise your notes and/or create a summary of your notes (from readings, classroom lectures, etc.)? 
Write about what you have read, reflecting on a paragraph, section, chapter, or unit? 
Write a summary sentence in order to synthesize a passage from a text? 


Write in journals or logs reflecting on what you have been learning in your classes or how you 
are doing and/or what your goals are? 


Write notes grouped into three categories/columns of questions, facts, and steps? 
Use paper folded a few times to help you organize and record information into categories? 


Plan your essays before writing, by first formulating and stating a clear thesis and organizing 
details and facts to support that thesis? 


Scale: 1 = never or rarely, 2 = occasionally (less than once a month), 3= once or twice a month, 4 = 


several times a month to weekly, 5 = more than once a week 


Engagement in School 


Tries hard and finds school interesting. The value of this construct was determined by calculating 


the average of the survey item responses (seven items, Cronbach’s alpha=0.85). 


How much do you agree with these statements about your current school experiences? 


You pay attention in class. 
You work very hard on your school work. 


There is a strong connection between what you are studying in school and what you want to study 
in college. 


There is a strong connection between what you are studying in school and what you want to do 
for a career. 


You think what you are learning in class is interesting. 
You understand what is expected of you in class and on your assignments. 


You try as hard as you can in school. 


Scale: 1 = not at all true, 2 = not really true, 3 = somewhat true, 4 = mostly true, 5 = always true 
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Focuses on long-term objectives. The value of this construct was determined by inverting the sur- 
vey responses and then calculating the average of the survey item responses (four items, Cronbach’s 
alpha = 0.77). 


For each of the following statements, indicate how much they describe you. 


e New ideas and projects sometimes distract you from previous ones. 
e You have been obsessed with a certain idea or project for a short time but later lost interest. 
e You often set a goal but later choose to pursue a different one. 


e You often have difficulty maintaining your focus on projects that take more than a few months 
to complete. 


Scale: 1 = not at all like you, 2 = not much like you, 3 = somewhat like you, 4 = mostly like you, 5 = 
very much like you 


Diligent and a hard worker. The value of this construct was determined by calculating the average 
of the survey item responses (three items, Cronbach’s alpha = 0.83). 


For each of the following statements, indicate how much they describe you. 


e You are a hard worker. 
e You finish whatever you begin. 
e You are diligent. 


Scale: 1 = not at all like you, 2 = not much like you, 3 = somewhat like you, 4 = mostly like you, 5 = 
very much like you 


Confident of ability at school. The value of this construct was determined by calculating the aver- 
age of the survey item responses (four items, Cronbach’s alpha = 0.94). 


Please indicate how true each statement is for you. 


e You feel you are confident in your ability to learn. 

e You are capable of learning the material in your classes. 

e You are able to achieve your educational goals. 

e You are able to meet the challenge of performing well in your classes. 


Scale: 1 = not at all true, 2 = not really true, 3 = somewhat true, 4 = mostly true, 5 = always true 
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Postsecondary Awareness and Expectations 


Has a plan for college. The value of this construct was determined by calculating the average of the 
survey item responses (four items, Cronbach’s alpha = 0.87). 


How true is each of these statements about you right now? 

e You have a good idea of what level of education is required for the kind of career you want. 
e You have a good idea of what kind of career you want to have. 

e You believe you will be able to get the kind of career you want. 


e You plan on going to college. 
Scale: 1 = not at all true, 2 = not really true, 3 = somewhat true, 4 = mostly true, 5 = always true 


Had opportunities to learn about college. The value of this construct was determined by calculating 
the average of the survey item responses (nine items, Cronbach’s alpha = 0.86). 


Have you ever... 

e Learned about what level of education is needed for various careers? 

e Visited a 4-year college, community college, or career-technical school campus? 
e Talked with a parent or other family member about college? 


e Discussed with anyone at your school what types of careers would suit your interests, talents, 
and skills? 


e Discussed with anyone at your school how to decide which college to attend? 
e Discussed with anyone at your school your chances of being accepted at different types of colleges? 


e Discussed with anyone at your school what ACT/SAT scores you need to get into the colleges you 
want to attend? 


e Discussed with anyone at your school how to pay for college? 
Scale: 0 = no, 1 = yes 
Planned level of degree attainment. The value of this construct corresponds to a single survey item. 


As things stand now, how far do you think you will go with your education? 


Scale: 1 = You probably won’t graduate high school, 2 = you will get a GED, 3 = you will graduate 
from high school and get a high school diploma, 4 = you will get a license or a certificate from a 
community college or a technical/vocational school, 5 = you will get a two-year college degree, 6 = 
you will get a four-year college degree, 7 = you will get an advanced degree (master’s or doctorate) 
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School Environment and Rigor 


School focuses on success of all students. The value of this construct was determined by calculating 


the average of the survey item responses (seven items, Cronbach’s alpha = 0.88). 


How much do you agree with the following statements? At your school... 


Teachers make sure that all students are planning for life after graduation. 

Teachers work hard to make sure that all students are learning. 

High school is seen as preparation for the future. 

All students are encouraged to continue their education after graduation from high school. 
Teachers pay attention to all students, not just the top students. 

Teachers work hard to make sure that all students stay in school. 


Teachers help students develop skills to be successful in life beyond high school. 


Scale: 1 = strongly disagree, 2= disagree, 3= agree, 4 = strongly agree 


School encourages all students to go to college.— The value of this construct was determined by 


calculating the average of the survey item responses (eight items, Cronbach’s alpha = 0.89). 


Please indicate how much you agree or disagree. 


You have been encouraged by at least one of your teachers to take honors, advanced placement, 
or dual enrollment classes. 


You have been encouraged by your guidance counselor to take honors, advanced placement, or 
dual enrollment classes. 


At least one of your teachers and guidance counselor has encouraged you to take classes that will 
be challenging to you. 


You know where to go to get the help you need to succeed in your classes, such as tutoring, home- 
work help, etc. 


At least one of your teachers has encouraged you to start planning for college. 
Your guidance counselor has encouraged you to start planning for college. 


At least one of your teachers has encouraged you to start planning for a career and life after high 
school. 
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e Your guidance counselor has encouraged you to start planning for a career and life after high school. 
Scale: 1 = strongly disagree, 2 = disagree, 3 = agree, 4 = strongly agree 


Collaborative and respectful environment. The value of this construct was determined by calculat- 
ing the average of the survey item responses (five items, Cronbach’s alpha = 0.81). 


Please indicate how much you agree or disagree. 


e Teachers and students show respect for different cultures, backgrounds, beliefs, and opinions. 
e Teachers and students treat each other with courtesy and respect. 

e Students in your classes help each other with their homework. 

e Students in your classes work well in groups. 

e Students in your classes listen to each other during class discussions. 

Scale: 1 = strongly disagree, 2 = disagree, 3 = agree, 4 = strongly agree 


Other students try hard at school. The value of this construct was determined by calculating the 
average of the survey item responses (four items, Cronbach’s alpha = 0.89). 


At your school, how many students in your grade... 


e Feel it is important to come to school every day? 
e Feel it is important to pay attention in class? 

e Think doing homework is important? 

e Try hard to get good grades? 


Scale: 1 = none, 2 = some, 3 = most, 4 = all 


SUPPLEMENTARY ANALYSES OF STAFF AND STUDENT SURVEY 
DATA 


Table 4.1 in Chapter 4 shows the composite staff survey findings for each of the key WICOR strate- 
gies across all the schools. Appendix Table B.3 breaks out the Year 3 scores for each of the schools. 
The table shows that many of the schools saw considerable growth in the Writing and Organization 
strategies by the third year. Most schools did not see statistically significant growth in Inquiry. 
The outcomes of a few specific schools bolstered the Collaboration and Reading scores, while other 
schools did not show statistically significant differences. On all measures, middle school staff re- 
ported higher scores than high school staff. 
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Appendix Table B.4 displays the school-level findings for the third implementation year on the 
measures of academic rigor, college preparation, and collaborative communication and vertical 
alignment. Almost all schools showed positive growth in staff members’ reporting of collaboration 
and vertical alignment. While schools tended to have similar scores on how staff members rated the 
school’s academic rigor, the level of growth on this measure varied across schools. In all the schools, 
average responses to the question of whether all students can be successful in college are between 
agreement and disagreement, suggesting a mixed response across all schools. 


Table 5.1 in Chapter 5 displays the average student outcomes from the student survey across all 
tenth-grade students. Appendix Table B.5 displays the same findings but only includes those stu- 
dents participating in the AVID elective. AVID elective students in both groups (those who had just 
started the program and those who had been participating in the program for up to three years) 
had higher average scores across the measures than the full sample of students. The AVID elective 
students in the third year of implementation did tend to report having more opportunities to learn 
about college than their counterparts at the beginning of the first year. They also agreed more with 
the idea that their school encouraged all students to go to college. 
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APPENDIX TABLE B.4 
AVID Core Values and Collaboration as Measured by the Staff Survey, by 


School, Year 3 
Academic Rigor College Preparationa Collaboration 
School Year3 Difference Year 3 Difference Year3 Difference 
All schools 3.11 0.115*** 2.42 0.193*** 2.33 0.348*** 
Middle school A 3.11 O23" 2.33 0.350 2.23 0.493*** 
High school A 3.18 0.033 2.24 0.203 2.10 0.353** 
Middle school B 3.36 0.248* 2.58 0.306 2.63 1.060*** 
High school B 3.34 Q,320*** 2.51 0.147 2.18 0.500*** 
Middle school C 3.05 0.156 2.63 0.264 2.49 0.442*** 
High school C 2.92 0.168 2.26 -0.204 2:51 0.114 
Middle school D 2.79 -0.138 2.42 0.398** 2.311 0.502*** 
High school D 3.14 0.044 2.43 0.143 2.21 -0.182 
Range 0.57 0.458 0.38 0.602 0.54 1.243 


SOURCE: Surveys of teachers, administrators, and guidance counselors administered in May 2013 and May 2016. 


NOTES: Across eight study schools, there were 427 participants in the spring before implementation and 421 
participants in Year 3. 


“Academic Rigor” and “College Preparation” were measured on a scale of 1 to 4, where 1 = strongly disagree and 4 
= strongly agree. “Collaboration” was measured on a scale of 1 to 6, where 1 = never and 6 = daily. 


The “Difference” columns display the difference between the score at the end of the third implementation year and 
the pre-score measured before implementation. A two-tailed t-test was used for all statistical tests presented in this 
table. Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; * = 10 percent. 


Data are missing for no more than 17 percent of respondents on any measure. 
Rounding may cause slight discrepancies in calculating sums and differences. 


This measure includes only one survey item: Staff members reported that all students at their school can be 
successful in college. 
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APPENDIX TABLE B.5 


Student Survey Outcomes: Tenth-Grade Students in AVID Elective, After Three Years 
of the AVID College Readiness System (ACRS) and Alignment Activities 


Estimated 
Composite Measure Fall 2013 Fall 2016 Difference P-Value 
Use of learning skills 
Frequency of use of reading and study strategies (1-4)? 2.60 2.59 -0.01 0.948 
Frequency of use of critical thinking in school (1-5)° 3.97 3.80 -0.17 0.353 
Frequency of use of study habits (1-5)° 3.73 3/05 -0.19 0.337 
Students report using a three-ring binder (%) 91.2 S73 6.1 0.213 
Students report using a calendar to track work (%) 85.1 83.2 -1.9 0.775 
Engagement in school* 
Tries hard and finds school interesting (1-5) 3.12 3.62 -0.11 0.478 
Focuses on long-term objectives (1-5) 3.40 3.22 -0.18 0.293 
Is diligent and a hard worker (1-5) 3.92 3.86 -0.06 0.717 
Is confident of ability at school (1-5) 3.96 3.93 -0.04 0.838 
Postsecondary awareness and expectations 
Has a plan for college (1-5)° 4.09 3:99 -0.10 0.535 
Had opportunities to learn about college (0-1)¢ 0.65 0.76 0.11 0.066* 
Planned level of degree attainment (1-7) 5.53 5.45 -0.07 0.812 
School environment and academic rigor 
School focuses on success of all students (1-4)! 2.93 2.97 0.04 0.764 
School encourages all students to go to college (1-4)* 2.83 3.05 0.22 0.089* 
Environment is collaborative and respectful (1-4)! 2.92 2.95 0.02 0.828 
Other students try hard at school (1-4)? 2.70 2.10 0.00 0.986 
Students report math class was difficult but manageable (%) 37.1 55.5 18.4 0.050** 
Students report English class was difficult but manageable 26.4 39.9 13:5. 0.133 
(%) 
Number of students 246 


(continued) 
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APPENDIX TABLE B.5 (continued) 


SOURCE: Surveys of students administered in fall 2013 and fall 2016. 


NOTES: Across four study high schools, 246 tenth-grade students in the AVID elective class participated in the survey, including 115 
students in the 2013 survey administration and 131 students in the 2016 survey administration. 


The “Fall 2013” score is regression adjusted using the mean covariate values for “Fall 2016” as the basis for adjustment. 


A two-tailed t-test was used for all statistical tests presented in this table. Asterisks after the “P-Value” column represent the statistical 
significance of the difference between Fall 2013 and Fall 2016. Statistical significance levels are indicated as follows: *** = 1 percent; ** 
= 5 percent; * = 10 percent. 


Rounding may cause slight discrepancies in calculating sums and differences. 

Data are missing for no more than 11 percent of respondents on any measure. 

This measure includes several items using the following scale: 1 = rarely, 4 = almost always. 

>This measure includes several items using the following scale: 1 = never or rarely, 5 = more than once a week. 

‘This measure includes several items using the following scale: 1 = not at all, 5 = always or very much. 

‘This measure includes several items using the following scale: 0 = no, 1 = yes. 

‘This measure uses the following scale: 1 = not graduate high school, 7 = get advanced degree (e.g., master’s, PhD). 
‘This measure includes several items using the following scale: 1 = strongly disagree, 4 = strongly agree. 


‘This measure includes several items using the following scale: 1 = none, 4 = all. 
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APPENDIX 


Student Academic Outcomes Supplemental 
Materials 


This appendix includes supplemental materials for the student academic outcomes discussed in 
Chapter 5. These analyses use students’ administrative records to explore the promise of the AVID 
College Readiness System (ACRS) and alignment activities to affect student outcomes, including 
advanced course credit earning, overall academic performance, educational attainment, and school 
persistence. The first section describes the analyses and compares the characteristics of the students 
during the years before program implementation with the characteristics of those students during 
implementation. The second section looks at supplemental analyses, including outcomes in the first 
and second years of implementation, outcomes by grade, and outcomes by high school. 


ANALYTIC METHOD AND CHARACTERISTICS OF STUDENTS IN 
THE ANALYSIS SAMPLE 


As noted in the report, this study is nonexperimental. The analysis is a comparison of the combined 
average academic outcomes for students during the three years before program implementation (the 
2010-2011, 2011-2012, and 2012-2013 school years) with the students’ average academic outcomes 
during the third year of implementation. Since the analysis combines the outcomes of four high 
schools and ninth- through twelfth-grade students in each of those schools, a multilevel model was 
used so that student outcomes are being compared within schools and grades and then averaged 
across the included schools and grades. To control for the differences in the student populations 
before and during program implementation, the analysis includes covariates for race and ethnicity 
and special education, as well as gender and each student’s proficiency level (1 to 5) on the Florida 
State standardized reading and math tests during sixth grade, which took place before the students 
participated in the program. Eligibility for free or reduced-price lunch was not included. This measure 
is somewhat problematic because two of the school districts in the study adopted the Community 
Eligibility Provision (CEP) during the study years. The CEP is a federal program for districts with 
high levels of low-income students that removes the need to collect applications from each student’s 
family and allows schools to provide breakfast and lunch for the entire student body. For this reason, 
students in the years before the study began are more likely to have up-to-date eligibility informa- 
tion in comparison with students in the program during the third year of the study, regardless of 
their actual eligibility. English language learner status was also not included because only about 2 
percent of students in these schools were designated with that status. 


Appendix Table C.1 compares the characteristics of the students in each of these two groups. As 
can be seen from the table, the two groups of students are similar to each other on many of the 
measures. But there are some differences between the two groups: There are more Hispanic students 
and fewer white and Asian students in the third year of implementation than in the years before 
implementation, and there is a higher percentage of special education students in the third year of 
implementation compared with the years before implementation. The study team also compared the 
same baseline characteristics for eighth-grade students. Similar to the high school students, there is 
a higher percentage of Hispanic students by the third year of implementation than in the years before 
implementation, and a smaller percentage of both white and black students (not shown in table). 
These differences suggest a slight change in the population of students during the years of the study, 
with a higher percentage of minority and high-needs students at the schools during the program 
years than there had been in the years before the study that are being used for the comparison. As 
noted above, these changes are controlled for in the analysis. 
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APPENDIX TABLE C.1 


Characteristics of Students in the Analytic Sample, Before and During 
the Third Year of the AVID College Readiness System (ACRS) and 
Alignment Activities 


Pre-AVID Year3 Estimated 
Characteristic Years (2015-2016) Difference P-Value 
Eligible for free/reduced-price lunch (%) 66.9 63.8 -3.1 0.223 
Race and ethnicity (%) 

Black 14.0 14.0 0.0 0.941 

Hispanic 39.6 44.7 5.1 0.000*** 

Asian 12 0.8 -0.4 0.046** 

White 41.6 37.4 -4.2 0.000*** 

Other 330 34 -0.5 0.380 
Gender (%) 

Male 51.2 50.4 -0.7 0.490 
English language learners (%) 2.6 2.0 -0.7 0.112 
Special education status (%) 11.2 14.0 2.8 0.007*** 
State assessment achievement level 
(1-5, 3 = passing, 6th grade) 

English Language Arts 2.60 2.59 -0.01 0.770 

Math 2.45 2.48 0.03 0.603 
Number of students 17,669 


SOURCE: MDRC calculations are based on student records obtained from school districts for 2010-2011 through 
2012-2013 (pre-AVID years) and 2015-2016 (Year 3). 


NOTES: Across four study high schools, 12,942 students were included in the pre-AVID implementation sample, 
which includes students in the study high schools during the three years before AVID implementation, and 4,727 
students were included in the Year 3 sample, which includes all students in the study high schools during the 2015- 
2016 school year. 


The “Pre-AVID Years” column displays the average scores for all students in ninth through twelfth grades during 
the three years before the first year of implementation, and the “Year 3” column displays the average scores during 
the third year of implementation of the ACRS and alignment activities. 


A two-tailed t-test was used for all statistical tests presented in this table. Asterisks after the “P-Value” column 
represent the statistical significance of the difference between Year 3 and the pre-AVID years. Statistical 
significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; * = 10 percent. 


Rounding may cause slight discrepancies in calculating sums and differences. 


Data are missing for no more than 1 percent of respondents on any measure except for the state English 
language arts achievement level, the state math achievement level, and eligibility for free/reduced-price lunch. On 
these measures, 22 percent of data are missing for the students after three years of implementation, and 25 percent 
of data are missing for the pre-AVID implementation group. 
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SUPPLEMENTAL STUDENT OUTCOMES FINDINGS 


Table 5.2 in Chapter 5 displays the student outcomes for students in the third year of implementa- 
tion. This section includes the findings for similar outcomes during the first and second years of 
implementation. This section also includes findings by grade level (ninth through twelfth grade) 
and findings for students attending each of the four high schools participating in the program and 
the study. 


Appendix Table C.2 shows similar outcome measures as Table 5.2 but for students after the first 
year of implementation. This table includes only ninth- and tenth-grade students because in the 
high schools, only ninth- and tenth-grade students participated in the program during the first 
year. Appendix Table C.3 displays similar outcomes for students in the second year of implementa- 
tion. This table includes outcomes for ninth-, tenth-, and eleventh-grade students, as the program 
added eleventh grade in the second year and was then targeting ninth-, tenth- and eleventh-grade 
students at the high schools. The Florida state assessment in reading was not included in this table 
because this was the first year that a new test was administered across the state and, during this 
year, proficiency levels were not provided. 


Appendix Table C.4 includes the same sample of students as in Table 5.2, but includes the findings 


by grade level (ninth through twelfth grade). Similarly, Appendix Table C.5 also includes the same 
sample as in Table 5.2 but includes the findings for each high school separately. 
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APPENDIX TABLE C.2 


Student Outcomes in the First Year of the AVID College Readiness System (ACRS) and 
Alignment Activities: Ninth- and Tenth-Grade Students Combined 


Pre-AVID Year1 Estimated 
Outcome Measure Years (2013-2014) Difference P-Value 
Credits earned in advanced course work? 
Earned at least 1 credit in an advanced course (%) 314 35.1 4.0 0.003*** 
Total advanced credits earned 0.76 0.82 0.06 0.254 
Core advanced credits earned? 0.70 0.79 0.09 0.075* 
Academic performance 
Grade point average (0-4) 2.39 2.26 -0.13 0.022** 
State English Language Arts achievement level 2.32 2.41 0.09 0.183 
(1-5, 3 = passing)° 
Educational attainment 
Total credits earned 4.98 4.95 -0.03 0.800 
Core credits earned? Salih 3.16 -0.01 0.903 
Promotion to the next grade (%) 71.6 74.5 29 0.204 
High school persistence 
Retention in school through the school year (%) 87.4 90.0 2.6 0.131 
Sample size 9,952 


SOURCE: MDRC calculations are based on student records obtained from school districts for 2010-2011 through 2012-2013 (pre-AVID years) 
and 2013-2014 (Year ‘). 

NOTES: Across four study high schools, 7,449 students were included in the pre-AVID implementation sample, which includes students in the 
study high schools during the three years before AVID implementation, and 2,503 students were included in the post sample, which includes all 

students in ninth through tenth grades in the study high schools during the 2013-2014 school year. 


The “Pre-AVID Years” column displays the average scores for all students in ninth through twelfth grades during the three years before the 
first year of implementation, and the “Year 1” column displays the average scores at the end of the first year of implementation of the ACRS and 
alignment activities. 


A two-tailed t-test was used for all statistical tests presented in this table. Asterisks after the “P-Value” column represent the statistical 
significance of the difference between Year 1 and pre-AVID years. Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 
percent; * = 10 percent. 


Rounding may cause slight discrepancies in calculating sums and differences. 


Data are missing for no more than 4 percent of respondents on any measure except for the state English Language Arts achievement level. 
For the students in the first year of implementation, 10 percent of data for reading achievement are missing, and for the pre-AVID implementation 
group, 21 percent of the data for reading achievement are missing. 


*Advanced course work consists of honors, dual enrollment, and Advanced Placement courses. 
Core courses include all English/language arts, math, science, and social studies courses. 


°Starting in spring 2015, the Florida state assessment changed from the Florida Comprehensive Assessment Test to the Florida Standards 
Assessment. The achievement levels for these tests are not equivalent, and some caution should be taken when interpreting these findings. 
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APPENDIX TABLE C.3 


Student Outcomes in the Second Year of the AVID College Readiness System (ACRS) and 
Alignment Activities: Ninth, Tenth, and Eleventh Grades Combined 


Pre-AVID Year 2 Estimated 

Outcome Measure Years (2014-2015) Difference P-Value 
Credits earned in advanced course work? 

Earned at least 1 credit in an advanced course (%) 29.1 36.2 71 0.000*** 

Total advanced credits earned 0.80 0.90 0.11 0.063* 

Core advanced credits earned? 0.71 0.83 0.12 0.014** 
Academic performance 

Grade point average (0-4) 2.43 2.43 0.00 0.973 
Educational attainment 

Total credits earned 5.07 5.06 -0.01 0.944 

Core credits earned> 3:29 3:25 0.00 0.958 

Promotion to the next grade (%) 74.1 78.4 4.2 0.034** 
High school persistence 

Retention in school through the school year (%) 87.7 90.3 2:1 0.107 
Sample size 13,820 


SOURCE: MDRC calculations are based on student records obtained from school districts for 2010-2011 through 2012-2013 (pre-AVID years) 
and 2014-2015 (Year 2). 


NOTES: Across four study high schools, 10,276 students were included in the pre-AVID implementation sample, which includes students in the 
study high schools during the three years before AVID implementation, and 3,544 students were included in the Year 3 sample, which includes all 
students in ninth through eleventh grades in the study high schools during the 2014-2015 school year. 


The “Pre-AVID Years” column displays the average scores for all students in ninth through twelfth grades during the three years before the first 
year of implementation, and the “Year 2” column displays the average scores at the end of the second year of implementation of the ACRS and 
alignment activities. 


A two-tailed t-test was used for all statistical tests presented in this table. Asterisks after the “P-Value” column represent the statistical 
significance of the difference between Year 2 and the pre-AVID years. Statistical significance levels are indicated as follows: *** = 1 percent; ** = 
5 percent; * = 10 percent. 


Rounding may cause slight discrepancies in calculating sums and differences. 

Data are missing for no more than 4 percent of respondents on any measure. 

Advanced course work consists of honors, dual enrollment, and Advanced Placement courses. 
Core courses include all English/language arts, math, science, and social studies courses. 
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APPENDIX TABLE C.4 


Student Outcomes in the Third Year of the AVID College Readiness System (ACRS) 


and Alignment Activities, by Grade 


Pre-AVID Year 1 Estimated 
Outcome Measure Years (2013-2014) Difference P-Value 
Credits earned in advanced course work? 
Earned at least 1 credit in an advanced course (%) 
9th grade 23:9 28.6 4.7 0.025** 
10th grade 26.4 35.6 9.2 0.003*** 
11th grade 35.7 45.5 9.8 0.013** 
12th grade 38.8 48.4 9:5 0.014** 
Total advanced credits earned 
9th grade O57 0.68 0.11 0.193 
10th grade 0.74 0.93 0.19 0.096* 
11th grade 1.13 1.27 0.14 0.313 
12th grade 1.24 1.19 -0.05 0.673 
Core advanced credits earned? 
9th grade 0.54 0.68 0.14 0.094* 
10th grade 0.66 0.86 0.20 0.042** 
11th grade 0.95 141 0.16 0.116 
12th grade 0.93 0.95 0.02 0.805 
Dual enrollment college credits earned 
9th grade 0.50 0.36 -0.14 0.174 
10th grade 0.77 0:52 -0.25 0.050** 
Honors and Advanced Placement credits earned 
9th grade 0.82 1.04 0.22 0.038** 
10th grade 0.68 0.81 0.13 0.161 
Academic performance 
Grade point average (0-4) 
9th grade 2.30 2.34 0.04 0.605 
10th grade 2.31 2:39 -0.04 0.626 
11th grade 2.63 2.69 0.06 0.412 
12th grade 2:94 3.05 0.14 0.088* 
State English Language Arts achievement level* 
(1-5, 3 = passing) 
9th grade 2.24 2:27 0.03 0.622 
10th grade 2.24 2.20 -0.04 0.733 
Educational attainment 
Total credits earned 
9th grade 4.65 4.91 0.27 0.188 
10th grade 5.15. 5.07 -0.08 0.588 
11th grade ba Br | 5.19 -0.12 0.443 
12th grade 5:55 5.37 -0.17 0.132 


(continued) 
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APPENDIX TABLE C.4 (continued) 


Pre-AVID Year 1 Estimated 
Outcome Measure Years (2013-2014) Difference P-Value 
Core credits earned? 
9th grade 2.68 2.85 0.17 0.321 
10th grade 3.53 3.43 -0.09 .0590 
11th grade 3.70 3.64 -0.07 0.599 
12th grade 3.24 3.99 0.09 0.341 
Promotion to the next grade (%) 
9th grade 67.8 77.6 9.8 0.012** 
10th grade 72.4 74.9 2.4 0.494 
11th grade 80.9 83.3 2 0.535 
12th grade 92.2 89.9 -2.3 0.429 
High school persistence 
Retention in school through the school year (%) 
9th grade 85.5 86.8 1:3 0.586 
10th grade 87.2 88.0 0.8 0.762 
11th grade 88.1 87.3 -0.9 0.793 
12th grade 94.6 92.4 -2.1 0.242 
Sample size 17,669 


SOURCE: MDRC calculations are based on student records obtained from school districts for 2010-2011 through 2012-2013 (pre-AVID years) 
and 2015-2016 (Year 3). 


NOTES: Across four study high schools, 12,942 students were included in the pre-AVID implementation sample, which includes students in the 
study high schools during the three years before AVID implementation, and 4,727 students were included in the Year 3 sample, which includes 
all students in the study high schools during the 2015-2016 school year. 


The “Pre-AVID Years” column displays the average scores during the three years before the first year of implementation, and the “Year 3” 
column displays the average scores at the end of the third year of implementation of the ACRS and alignment activities. 


A two-tailed t-test was used for all statistical tests presented in this table. Asterisks after the “P-Value” column represent the statistical 
significance of the difference between the Year 3 and the pre-AVID years. Statistical significance levels are indicated as follows: *** = 1 percent; 
* = § percent; * = 10 percent. 


Rounding may cause slight discrepancies in calculating sums and differences. 


Data are missing for no more than 5 percent of respondents on any measure except for the state English Language Arts achievement level. 
On that measure, 14 percent of the data are missing for the students after three years of implementation, and 21 percent of the data are missing 
for the pre-AVID implementation group. Also, in the pre-AVID implementation group, most data were missing for tenth-grade students in one 
district during 2010-2011 and 2011-2012, so only 2012-2013 data were included for that district and grade. 


One district did not provide data on dual enrollment courses taken and passed. For this district, dual enrollment courses are not included in 
the measures of advanced credits earned, and the district is not included in the measure of dual enrollment credits earned. 


*Core courses include all English/language arts, math, science, and social studies courses. 


°Starting in spring 2015, the Florida state assessment changed from the Florida Comprehensive Assessment Test to the Florida Standards 
Assessment. The achievement levels for these tests are not equivalent, and some caution should be taken when interpreting these findings. 
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APPENDIX TABLE C.5 


Student Outcomes in the Third Year of the AVID College Readiness System (ACRS) and 
Alignment Activities, by High School Attended 


Pre-AVID Year3 Estimated 
Outcome Measure Years (2015-2016) Difference P-Value 
Credits earned in advanced course work 
Earned at least 1 credit in an advanced course (%) 
High school A 30.8 42.1 11.3. 0.001*** 
High school B Bole. 44.7 11.6 0.001*** 
High school C 30.6 32.1 15 0.499 
High school D 27.5 36.8 9.4 0.028** 
Total advanced credits earned 
High school A 0.86 1.28 0.42 0.002*** 
High school B 0.98 1.16 0.19 0.050* 
High school C 1.03 0.83 -0.20 0.109 
High school D 0.67 0.77 0.10 0.179 
Core advanced credits earned? 
High school A 0.71 1.05 0.35 0.0017" 
High school B 0.82 1.02 0.20 0.030** 
High school C 0.80 0.75 -0.05 0.646 
High school D 0.67 0.77 0.10 0.179 
Dual enrollment college credits earned 
High school A 0.59 0.69 0.10 0.371 
High school B 0.58 0.37 -0.21 0:022*" 
High school C 0:75 0.30 -0.46 0.001*** 
High school D 
Honors/Advanced Placement credits earned (11th—12th grades) 
High school A 0.57 0.97 0.40 0.032** 
High school B 1.02 124 0.19 0.054* 
High school C 0.80 0.78 -0.02 0.898 
High school D 0.73 0.86 O13 0.258 
Academic performance 
Grade point average (0-4) 
High school A 2.62 2.67 0.04 0.399 
High school B 2.66 2.86 0.19 0.002*** 
High school C 2.51 2.49 -0.02 0.847 
High school D 2.40 2.36 -0.04 0.643 
State English Language Arts achievement level’ (1-5, 3 = passing) 
High school A 2.19 2:16 -0.03 0.833 
High school B 2.26 2.28 0.02 0.842 
High school C 1.96 2.09 O13 0.063* 
High school D 2.44 2.38 -0.06 0.695 


(continued) 
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APPENDIX TABLE C.5 (continued) 


Pre-AVID Year3 Estimated 
Outcome Measure Years (2015-2016) Difference P-Value 
Educational attainment 
Total credits earned 
High school A 5.70 6.02 0.32 0.116 
High school B 6.00 6.18 0.19 0.065* 
High school C 6.05 5.53 -0.52 0.001*** 
High school D 3.56 3.38 -0.17 0.135 
Core credits earned? 
High school A 3.06 3.47 0.41 0.001*** 
High school B 3.24 3.51 0.28 0.009*** 
High school C 3:33 2.85 -0.48 0.001*** 
High school D 3.06 3.38 -0.17 0.135 
Promotion to the next grade (%) 
High school A 72.6 77.0 4.4 0.250 
High school B 72.1 77.9 5.8 0.007*** 
High school C 76.3 78.3 2.0 0.620 
High school D 88.0 86.6 -1.4 0.715 
High school graduation 
High school A 0.83 0.83 0.00 0.956 
High school B 0.87 0.89 0.02 0.570 
High school C 0.83 0.75 -0.09 0.418 
High school D 0.89 0.88 -0.01 0.849 
High school persistence 
Retention in school through the school year (%) 
High school A 87.3 88.9 rg 0.279 
High school B 87.5 93.2 HA 0.002*** 
High school C 89.9 83.4 -6.5 0.000*** 
High school D 90.8 89.7 -1.1 0.776 
Sample size 17,669 


(continued) 
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APPENDIX TABLE C.5 (continued) 


SOURCE: MDRC calculations are based on student records obtained from school districts for 2010-2011 through 2012-2013 (pre-AVID years) 
and 2015-2016 (Year 3). 


NOTES: Across four study high schools, 12,942 students were included in the pre-AVID implementation sample, which includes students in the 
study high schools during the three years before AVID implementation, and 4,727 students were included in the Year 3 sample, which includes all 
students in the study high schools during the 2015-2016 school year. 


The “Pre-AVID Years” column displays the average scores for all students in ninth through twelfth grades during the three years before the 
first year of implementation, and the “Year 3” column displays the average scores at the end of the third year of implementation of the ACRS and 
alignment activities. 


A two-tailed t-test was used for all statistical tests presented in this table. Asterisks after the “P-Value” column represent the statistical 
significance of the difference between the Year 3 and the pre-AVID years. Statistical significance levels are indicated as follows: *** = 1 percent; 
* = § percent; * = 10 percent. 


Rounding may cause slight discrepancies in calculating sums and differences. 


Data are missing for no more than 6 percent of respondents on any measure except for the state English Language Arts achievement level. 
On that measure, no more than 23 percent of the data are missing for any one school for the students after three years of implementation, and no 
more than 38 percent of the data are missing for any one school for the pre-AVID implementation group. 


Core courses include all English/language arts, math, science, and social studies courses. 


Starting in spring 2015, the Florida state assessment changed from the Florida Comprehensive Assessment Test to the Florida Standards 
Assessment. The achievement levels for these tests are not equivalent, and some caution should be taken when interpreting these findings. 
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ABOUT MDRC 


MDRC IS A NONPROFIT, NONPARTISAN SOCIAL AND EDU- 
CATION POLICY RESEARCH ORGANIZATION DEDICATED TO 
learning what works to improve the well-being of low-income 
people. Through its research and the active communication of its 
findings, MDRC seeks to enhance the effectiveness of social and 
education policies and programs. 


Founded in 1974 and located in New York; Oakland, California; 
Washington, DC; and Los Angeles, MDRC is best known for 
mounting rigorous, large-scale, real-world tests of new and ex- 
isting policies and programs. Its projects are a mix of demon- 
strations (field tests of promising new program approaches) and 
evaluations of ongoing government and community initiatives. 
MDRC’s staff members bring an unusual combination of research 
and organizational experience to their work, providing expertise 
on the latest in qualitative and quantitative methods and on pro- 
gram design, development, implementation, and management. 
MDRC seeks to learn not just whether a program is effective but 
also how and why the program’s effects occur. In addition, it tries 
to place each project’s findings in the broader context of related 
research — in order to build knowledge about what works across 
the social and education policy fields. MDRC’s findings, lessons, 
and best practices are shared with a broad audience in the policy 
and practitioner community as well as with the general public and 
the media. 


Over the years, MDRC has brought its unique approach to an 
ever-growing range of policy areas and target populations. 
Once known primarily for evaluations of state welfare-to-work 
programs, today MDRC is also studying public school reforms, 
employment programs for ex-prisoners, and programs to help 
low-income students succeed in college. MDRC’s projects are 
organized into five areas: 


e Promoting Family Well-Being and Children’s Development 
e Improving Public Education 

e Raising Academic Achievement and Persistence in College 
e Supporting Low-Wage Workers and Communities 

e Overcoming Barriers to Employment 


Working in almost every state, all of the nation’s largest cities, 
and Canada and the United Kingdom, MDRC conducts its proj- 
ects in partnership with national, state, and local governments, 
public school systems, community organizations, and numerous 
private philanthropies. 


